- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes training-free inference-time guidance using feature statistics of a frozen generator's own high-scoring outputs. It ranks a generated pool, applies PCA to high-scoring feature tokens, and selects directions with the largest Jensen–Shannon divergence between high- and low-scoring tokens. During sampling, it selects patches that deviate most from the reference and minimizes a sliced-Wasserstein matching loss using normalized gradient updates. The paper reports non-negative average changes in four preference metrics across three text-to-image generators, improvements on eight personalized models, and human preferences favoring guided samples for two generators. A separate ImageNet pilot uses real-image references, DINOv3 features, and moment matching rather than the complete self-distilled method.
Critical strengths
Using the generator's own high-scoring samples provides a practical advantage because the reference can be reused and the target metric is queried only during offline ranking, without requiring differentiability. Table 3 reports non-negative average metric changes across three generators and four objectives under paired sampling. Table 2 further reports human preference win rates of 76.8% for SD3.5-Medium and 69.4% for PixNerd-T2I, excluding ties. Together, these results provide evidence that the average improvements extend beyond the optimized score. The ablations provide useful evidence for the design choices. In the SD3.5-Medium photo setting, Table 8 reports an ImageReward change of −0.096 when all patches are guided, compared with +0.090 when the most off-reference quarter is selected, both relative to vanilla sampling. Tables 6–7 also examine the feature space and matching objective. The analyses of content changes and borrowed-autoencoder limitations in Appendices J/K further clarify the method's operating conditions and failure modes.
Critical weaknesses
The method's competitiveness under a comparable compute budget remains unclear despite the improvements over vanilla sampling. The main experiments do not compare against best-of-N or an existing reward/distribution guidance method, despite discussing these alternatives. Reference construction has an offline cost, and online guidance requires feature-path differentiation and, where applicable, encoding and decoding. The paper does not report the latency, memory, or evaluation accounting needed to determine whether the gains justify this overhead. This is a substantive gap for a method motivated by the cost of repeated generation and selection. The average preference gains do not yet establish reliable preservation of individual prompt requirements or personalized identity. Appendix H provides useful measurements across all 3,200 paired samples, including CLIP-T, CLIP-I, and DINOv2, which support broad scene similarity and approximately stable average prompt alignment. However, these aggregate measures do not establish whether required objects, counts, or attributes remain correct, a limitation also acknowledged in Appendix J. Appendices J/K report ImageReward decreases on approximately 40% of SD3.5-Medium prompts and 46% of PixNerd-T2I prompts; these are reward-regression rates, not semantic-error rates. The prompt-match judgments in Tables 18–19 provide further evidence of content changes, but evaluate extreme success/failure subsets rather than a representative sample. A randomly sampled, constraint-level evaluation is therefore needed to estimate how often guidance compromises prompt requirements. Similarly, Table 4 and the selected DreamBooth galleries show preference improvements without directly quantifying preservation of the personalized subject's identity.
Questions for authors
- At the same total runtime budget, how does ManiGuide compare with best-of-N and an appropriate existing reward/distribution guidance baseline? Please report offline reference cost, amortization, per-image latency, peak memory, and generator evaluations, including how Algorithm 2's scheduler call uses or recomputes the velocity.
- Can you provide the distribution of per-prompt reward changes and prompt-faithfulness judgments on a randomly sampled evaluation subset? The extreme success/failure subsets in Tables 18–19 are informative, but do not estimate population-level semantic error. For DreamBooth, can you provide quantitative or blinded human subject-identity preservation results?
- How were the guidance strength, patch fraction, and reference configuration selected independently of evaluation? In particular, was the FLUX HPS-specific patch fraction fixed on held-out validation data? Please provide paired uncertainty across seeds and a random-patch or high-scoring-PCA-only control to distinguish the contribution of JS-based selection.
2 comments
Response to Reviewer StsD (2/2)
Selection of guidance strength, patch fraction and reference (Q3.1)
How were the guidance strength, patch fraction, and reference configuration selected independently of evaluation?
was the FLUX HPS-specific patch fraction fixed on held-out validation data?
See Common concern 4 for the hyperparameter selection. The FLUX HPS v2.1 patch fraction was selected by the same rule, on held-out data.
Paired uncertainty across seeds (Q3.2)
Please provide paired uncertainty across seeds
We ran ManiGuide on SD3.5-Medium with three independent seeds for each objective in Table 3 (ImageReward, Aesthetic, HPS v2.1, and PickScore). For each cell, we report the mean change from vanilla and a 95% paired bootstrap confidence interval over the 3,200 prompts.
| Target | ImageReward | Aesthetic | HPS v2.1 | PickScore |
|---|---|---|---|---|
| ImageReward | +0.072 [+0.065, +0.080] | +0.098 [+0.092, +0.103] | +0.0102 [+0.0099, +0.0106] | +0.138 [+0.128, +0.148] |
| Aesthetic | +0.043 [+0.035, +0.050] | +0.098 [+0.092, +0.103] | +0.0065 [+0.0061, +0.0068] | +0.112 [+0.102, +0.123] |
| HPS v2.1 | +0.062 [+0.055, +0.070] | +0.096 [+0.091, +0.101] | +0.0082 [+0.0079, +0.0085] | +0.103 [+0.093, +0.113] |
| PickScore | +0.062 [+0.055, +0.070] | +0.095 [+0.090, +0.100] | +0.0080 [+0.0077, +0.0083] | +0.114 [+0.104, +0.123] |
We added this table to App. N.1.
High-scoring-PCA-only control (Q3.3)
a random-patch or high-scoring-PCA-only control to distinguish the contribution of JS-based selection
With ImageReward as the objective and all else fixed, replacing JS-selected directions with the top 100 PCA directions from high-scoring reference tokens reduced gains by 0.011 on ImageReward, 0.024 on Aesthetic, 0.0010 on HPS v2.1, and 0.018 on PickScore. We added this control to App. M.6.
We hope this addresses the reviewer's concerns.
References:
[1] Shekhar et al. VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion. arXiv 2026.
[2] Jain et al. Diffusion Tree Sampling: Scalable Inference-Time Alignment of Diffusion Models. NeurIPS 2025.
[3] Bai et al. Qwen2.5-VL Technical Report. arXiv 2025.
Response to Reviewer StsD (1/2)
We thank the reviewer for the constructive feedback. We address each point below.
Comparisons with baselines, and cost (W1, Q1.1, Q1.2)
See Common concern 1 for comparisons with FKD, VASR [1], DTS [2] and DPS, Common concern 2 for the comparison with best-of-N, and Common concern 3 for offline reference construction and amortization, per-image latency, peak memory and generator evaluations.
Scheduler update (Q1.3)
Algorithm 2 reuses the original velocity vi for the scheduler update. It does not recompute the velocity after guidance.
At step i, the transformer predicts vi=vθ(zi,σi) at the original latent zi, just as in vanilla sampling. We detach vi to form the clean estimate x^0=zi−σisg(vi) and compute the guidance gradient through the frozen VAE feature path, without backpropagating through the transformer. After guidance updates the latent to zi′=zi−ηgi (Eq. 8), the scheduler takes an Euler step from zi′ using the original vi:
= z_i + (\sigma_{i+1} - \sigma_i)v_i - \eta g_i.$$ The next velocity $v_{i+1}$ is evaluated at the guided latent $z_{i+1}$ as part of the next denoising step. Guidance therefore affects subsequent predictions without adding any transformer evaluations. We added this explanation to App. A. ### Prompt requirements (W2, Q2.2) > A randomly sampled, constraint-level evaluation is therefore needed to estimate how often guidance compromises prompt requirements. We evaluated all 800 HPS v2.1 photo prompts with SD3.5-M, without selecting prompts based on reward changes. Qwen2.5-VL-7B [3] extracted 3,172 requirements before seeing any images, then checked each requirement separately in the vanilla and guided image for its prompt. In a manual check of 40 vanilla judgments, 37 agreed with our assessment and three were ambiguous. ManiGuide missed at least one requirement that vanilla satisfied on 9.9% of prompts, and satisfied at least one requirement that vanilla missed on 13.2% of prompts. Across all requirements, the error rate went from 12.0% to 11.1%, with no statistically significant increase overall or for any constraint type. | Constraint type | Requirements | Vanilla error | ManiGuide error | |---|---:|---:|---:| | Object | 1,722 | 6.8% | 6.3% | | Count | 125 | 28.0% | 29.6% | | Attribute | 340 | 14.7% | 11.2% | | Spatial | 802 | 17.5% | 16.1% | | Action / text | 183 | 21.9% | 21.9% | | **All** | **3,172** | **12.0%** | **11.1%** | We added this evaluation to App. O. ### Subject identity (W2, Q2.3) > For DreamBooth, can you provide quantitative or blinded human subject-identity preservation results? The table reports per-subject changes from vanilla in subject fidelity (CLIP-I and DINO) and prompt alignment (CLIP-T), with 500 prompts and 500 samples per method for each subject. Subject fidelity decreases slightly for all eight subjects. Prompt alignment stays nearly unchanged. | Metric | can | cat | dog 5 | dog 6 | teapot | cat 2 | dog 7 | robot toy | **Mean** | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | $\Delta$CLIP-I | -0.0032 | -0.0102 | -0.0099 | -0.0043 | -0.0034 | -0.0066 | -0.0071 | -0.0076 | **-0.0065** | | $\Delta$DINO | -0.0105 | -0.0092 | -0.0078 | -0.0072 | -0.0037 | -0.0089 | -0.0137 | -0.0025 | **-0.0079** | | $\Delta$CLIP-T | +0.0021 | +0.0019 | +0.0037 | +0.0020 | +0.0014 | +0.0005 | +0.0024 | -0.0008 | **+0.0017** | We added this table to App. P. ### Distribution of per-prompt reward changes (Q2.1) > Can you provide the distribution of per-prompt reward changes On the 800 HPS v2.1 photo prompts with SD3.5-M, ManiGuide's ImageReward change has a mean of +0.090 and a median of +0.05. The middle 50% of changes lie between -0.06 and +0.20, and the 5th and 95th percentiles are -0.37 and +0.70. Scores drop on about 40% of prompts (App. J). At thresholds of 0.1, 0.2, and 0.5 points, gains exceed the threshold on 40.2%, 25.8%, and 10.0% of prompts, while drops exceed it on 20.6%, 11.5%, and 3.5%, respectively. On the other hand, compared with the original vanilla sample, a second vanilla draw scores lower on 48.0% of prompts and drops by more than 0.5 points on 11.1%, versus 40% and 3.5% with one ManiGuide sample. Keeping the higher-scoring of two ManiGuide samples reduces the regression rate to 20.5%. We added this distribution to App. N.3.