acceptodds
Under review as a conference paper at ICLR 2027

ManiGuide: Training-Free Preference Alignment Across Diffusion Models

Abstract

Text-to-image generation has been advancing fast, yet the quality of a single sample still varies widely: users routinely draw many generations and cherry-pick the best one, since even a strong frozen model rarely produces its own best-case quality on a single try. Inspired by the Heeger–Bergen principle of distributional feature matching, we propose ManiGuide, a training-free inference-time guidance method that pushes a frozen generative model's typical output toward its own upper-bound quality as measured by any given metric, through patch-wise sliced-Wasserstein feature matching. Given a scoring metric, we first rank a pool of the model's own generation samples and use the top-ranked images to construct a reference manifold in deep-feature space for self-distillation. Then in generation, we identify patches whose feature distributions deviate most from the reference and guide them toward it using sliced-Wasserstein optimal transport. Across both pixel and latent image generative models, ManiGuide improves or preserves every metric. ManiGuide also generalizes beyond the base model to personalized models: on DreamBooth subject-driven fine-tunes, it turns each personalized model's own best generations into its guidance reference and improves preference metrics without additional training.

Primary area:
generative models
Keywords:
Diffusion Models, Image Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_StsDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes training-free inference-time guidance using feature statistics of a frozen generator's own high-scoring outputs. It ranks a generated pool, applies PCA to high-scoring feature tokens, and selects directions with the largest Jensen–Shannon divergence between high- and low-scoring tokens. During sampling, it selects patches that deviate most from the reference and minimizes a sliced-Wasserstein matching loss using normalized gradient updates. The paper reports non-negative average changes in four preference metrics across three text-to-image generators, improvements on eight personalized models, and human preferences favoring guided samples for two generators. A separate ImageNet pilot uses real-image references, DINOv3 features, and moment matching rather than the complete self-distilled method.

Critical strengths

Using the generator's own high-scoring samples provides a practical advantage because the reference can be reused and the target metric is queried only during offline ranking, without requiring differentiability. Table 3 reports non-negative average metric changes across three generators and four objectives under paired sampling. Table 2 further reports human preference win rates of 76.8% for SD3.5-Medium and 69.4% for PixNerd-T2I, excluding ties. Together, these results provide evidence that the average improvements extend beyond the optimized score. The ablations provide useful evidence for the design choices. In the SD3.5-Medium photo setting, Table 8 reports an ImageReward change of −0.096 when all patches are guided, compared with +0.090 when the most off-reference quarter is selected, both relative to vanilla sampling. Tables 6–7 also examine the feature space and matching objective. The analyses of content changes and borrowed-autoencoder limitations in Appendices J/K further clarify the method's operating conditions and failure modes.

Critical weaknesses

The method's competitiveness under a comparable compute budget remains unclear despite the improvements over vanilla sampling. The main experiments do not compare against best-of-N or an existing reward/distribution guidance method, despite discussing these alternatives. Reference construction has an offline cost, and online guidance requires feature-path differentiation and, where applicable, encoding and decoding. The paper does not report the latency, memory, or evaluation accounting needed to determine whether the gains justify this overhead. This is a substantive gap for a method motivated by the cost of repeated generation and selection. The average preference gains do not yet establish reliable preservation of individual prompt requirements or personalized identity. Appendix H provides useful measurements across all 3,200 paired samples, including CLIP-T, CLIP-I, and DINOv2, which support broad scene similarity and approximately stable average prompt alignment. However, these aggregate measures do not establish whether required objects, counts, or attributes remain correct, a limitation also acknowledged in Appendix J. Appendices J/K report ImageReward decreases on approximately 40% of SD3.5-Medium prompts and 46% of PixNerd-T2I prompts; these are reward-regression rates, not semantic-error rates. The prompt-match judgments in Tables 18–19 provide further evidence of content changes, but evaluate extreme success/failure subsets rather than a representative sample. A randomly sampled, constraint-level evaluation is therefore needed to estimate how often guidance compromises prompt requirements. Similarly, Table 4 and the selected DreamBooth galleries show preference improvements without directly quantifying preservation of the personalized subject's identity.

Questions for authors
  1. At the same total runtime budget, how does ManiGuide compare with best-of-N and an appropriate existing reward/distribution guidance baseline? Please report offline reference cost, amortization, per-image latency, peak memory, and generator evaluations, including how Algorithm 2's scheduler call uses or recomputes the velocity.
  2. Can you provide the distribution of per-prompt reward changes and prompt-faithfulness judgments on a randomly sampled evaluation subset? The extreme success/failure subsets in Tables 18–19 are informative, but do not estimate population-level semantic error. For DreamBooth, can you provide quantitative or blinded human subject-identity preservation results?
  3. How were the guidance strength, patch fraction, and reference configuration selected independently of evaluation? In particular, was the FLUX HPS-specific patch fraction fixed on held-out validation data? Please provide paired uncertainty across seeds and a random-patch or high-scoring-PCA-only control to distinguish the contribution of JS-based selection.
2 comments
AuthorsOpenReview ↗

Response to Reviewer StsD (1/2)

We thank the reviewer for the constructive feedback. We address each point below.

Comparisons with baselines, and cost (W1, Q1.1, Q1.2)

At the same total runtime budget, how does ManiGuide compare with best-of-N and an appropriate existing reward/distribution guidance baseline?

Please report offline reference cost, amortization, per-image latency, peak memory, and generator evaluations

See Common concern 1 for comparisons with FKD, VASR [1], DTS [2] and DPS, Common concern 2 for the comparison with best-of-N, and Common concern 3 for offline reference construction and amortization, per-image latency, peak memory and generator evaluations.

Scheduler update (Q1.3)

how Algorithm 2's scheduler call uses or recomputes the velocity

Algorithm 2 reuses the original velocity viv_i for the scheduler update. It does not recompute the velocity after guidance.

At step ii, the transformer predicts vi=vθ(zi,σi)v_i = v_\theta(z_i, \sigma_i) at the original latent ziz_i, just as in vanilla sampling. We detach viv_i to form the clean estimate x^0=zi−σi sg(vi)\hat{x}_0 = z_i - \sigma_i\,\mathrm{sg}(v_i) and compute the guidance gradient through the frozen VAE feature path, without backpropagating through the transformer. After guidance updates the latent to zi′=zi−ηgiz_i' = z_i - \eta g_i (Eq. 8), the scheduler takes an Euler step from zi′z_i' using the original viv_i:

= z_i + (\sigma_{i+1} - \sigma_i)v_i - \eta g_i.$$ The next velocity $v_{i+1}$ is evaluated at the guided latent $z_{i+1}$ as part of the next denoising step. Guidance therefore affects subsequent predictions without adding any transformer evaluations. We added this explanation to App. A. ### Prompt requirements (W2, Q2.2) > A randomly sampled, constraint-level evaluation is therefore needed to estimate how often guidance compromises prompt requirements. We evaluated all 800 HPS v2.1 photo prompts with SD3.5-M, without selecting prompts based on reward changes. Qwen2.5-VL-7B [3] extracted 3,172 requirements before seeing any images, then checked each requirement separately in the vanilla and guided image for its prompt. In a manual check of 40 vanilla judgments, 37 agreed with our assessment and three were ambiguous. ManiGuide missed at least one requirement that vanilla satisfied on 9.9% of prompts, and satisfied at least one requirement that vanilla missed on 13.2% of prompts. Across all requirements, the error rate went from 12.0% to 11.1%, with no statistically significant increase overall or for any constraint type. | Constraint type | Requirements | Vanilla error | ManiGuide error | |---|---:|---:|---:| | Object | 1,722 | 6.8% | 6.3% | | Count | 125 | 28.0% | 29.6% | | Attribute | 340 | 14.7% | 11.2% | | Spatial | 802 | 17.5% | 16.1% | | Action / text | 183 | 21.9% | 21.9% | | **All** | **3,172** | **12.0%** | **11.1%** | We added this evaluation to App. O. ### Subject identity (W2, Q2.3) > For DreamBooth, can you provide quantitative or blinded human subject-identity preservation results? The table reports per-subject changes from vanilla in subject fidelity (CLIP-I and DINO) and prompt alignment (CLIP-T), with 500 prompts and 500 samples per method for each subject. Subject fidelity decreases slightly for all eight subjects. Prompt alignment stays nearly unchanged. | Metric | can | cat | dog 5 | dog 6 | teapot | cat 2 | dog 7 | robot toy | **Mean** | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | $\Delta$CLIP-I | -0.0032 | -0.0102 | -0.0099 | -0.0043 | -0.0034 | -0.0066 | -0.0071 | -0.0076 | **-0.0065** | | $\Delta$DINO | -0.0105 | -0.0092 | -0.0078 | -0.0072 | -0.0037 | -0.0089 | -0.0137 | -0.0025 | **-0.0079** | | $\Delta$CLIP-T | +0.0021 | +0.0019 | +0.0037 | +0.0020 | +0.0014 | +0.0005 | +0.0024 | -0.0008 | **+0.0017** | We added this table to App. P. ### Distribution of per-prompt reward changes (Q2.1) > Can you provide the distribution of per-prompt reward changes On the 800 HPS v2.1 photo prompts with SD3.5-M, ManiGuide's ImageReward change has a mean of +0.090 and a median of +0.05. The middle 50% of changes lie between -0.06 and +0.20, and the 5th and 95th percentiles are -0.37 and +0.70. Scores drop on about 40% of prompts (App. J). At thresholds of 0.1, 0.2, and 0.5 points, gains exceed the threshold on 40.2%, 25.8%, and 10.0% of prompts, while drops exceed it on 20.6%, 11.5%, and 3.5%, respectively. On the other hand, compared with the original vanilla sample, a second vanilla draw scores lower on 48.0% of prompts and drops by more than 0.5 points on 11.1%, versus 40% and 3.5% with one ManiGuide sample. Keeping the higher-scoring of two ManiGuide samples reduces the regression rate to 20.5%. We added this distribution to App. N.3.
AuthorsOpenReview ↗

Response to Reviewer StsD (2/2)

Selection of guidance strength, patch fraction and reference (Q3.1)

How were the guidance strength, patch fraction, and reference configuration selected independently of evaluation?

was the FLUX HPS-specific patch fraction fixed on held-out validation data?

See Common concern 4 for the hyperparameter selection. The FLUX HPS v2.1 patch fraction was selected by the same rule, on held-out data.

Paired uncertainty across seeds (Q3.2)

Please provide paired uncertainty across seeds

We ran ManiGuide on SD3.5-Medium with three independent seeds for each objective in Table 3 (ImageReward, Aesthetic, HPS v2.1, and PickScore). For each cell, we report the mean change from vanilla and a 95% paired bootstrap confidence interval over the 3,200 prompts.

TargetImageRewardAestheticHPS v2.1PickScore
ImageReward+0.072 [+0.065, +0.080]+0.098 [+0.092, +0.103]+0.0102 [+0.0099, +0.0106]+0.138 [+0.128, +0.148]
Aesthetic+0.043 [+0.035, +0.050]+0.098 [+0.092, +0.103]+0.0065 [+0.0061, +0.0068]+0.112 [+0.102, +0.123]
HPS v2.1+0.062 [+0.055, +0.070]+0.096 [+0.091, +0.101]+0.0082 [+0.0079, +0.0085]+0.103 [+0.093, +0.113]
PickScore+0.062 [+0.055, +0.070]+0.095 [+0.090, +0.100]+0.0080 [+0.0077, +0.0083]+0.114 [+0.104, +0.123]

We added this table to App. N.1.

High-scoring-PCA-only control (Q3.3)

a random-patch or high-scoring-PCA-only control to distinguish the contribution of JS-based selection

With ImageReward as the objective and all else fixed, replacing JS-selected directions with the top 100 PCA directions from high-scoring reference tokens reduced gains by 0.011 on ImageReward, 0.024 on Aesthetic, 0.0010 on HPS v2.1, and 0.018 on PickScore. We added this control to App. M.6.

We hope this addresses the reviewer's concerns.

References:

[1] Shekhar et al. VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion. arXiv 2026.

[2] Jain et al. Diffusion Tree Sampling: Scalable Inference-Time Alignment of Diffusion Models. NeurIPS 2025.

[3] Bai et al. Qwen2.5-VL Technical Report. arXiv 2025.

Reviewer_14WXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The method. The paper proposes ManiGuide, a training-free, inference-time guidance method that aims to raise a frozen text-to-image model's typical output quality under a chosen metric.

  • Offline phase. It samples a pool of images from the model using LAION captions and ranks it with a preference metric: ImageReward, the LAION aesthetic predictor, HPS v2.1 or PickScore. It keeps the 16 best and 16 worst images. It then takes patch tokens from the model's own VAE-encoder features. It keeps the PCA directions of the best images' tokens that best separate best from worst (by Jensen–Shannon divergence), and stores the best images' token distribution along those directions.
  • Online phase. At each sampling step it decodes the clean estimate and re-encodes it. It then selects the fraction of patches that deviate most from the reference (25% by default) and takes a sphere-normalized gradient step on a patch-wise sliced-Wasserstein loss toward the reference.

Experiments.

  • Text-to-image. SD3.5-Medium, FLUX.1-dev and PixNerd-T2I on the 3,200-prompt HPS v2.1 benchmark. Mean gains are non-negative on all four metrics, and raters prefer the guided image 76.8% and 69.4% of the time against vanilla.
  • Personalization. Eight SDXL DreamBooth-LoRAs.
  • ImageNet. A 256 × 256 pilot that lowers FID for three class-conditional models.
  • Ablations. On the photo slice: reference source and size, feature space, matching objective and patch fraction.
Critical strengths
  1. A simple, broadly applicable design with a consistent effect that humans can see.

    • The metric is used only offline, to rank the model's own samples. Sampling never queries a reward model, and in principle the metric need not be differentiable.
    • Across two latent models (SD3.5-Medium, FLUX.1-dev) and one pixel-space model (PixNerd-T2I), no mean change in Table 3 is negative.
    • In a paired side-by-side study, raters prefer the guided image in 76.8% (SD3.5-M) and 69.4% (PixNerd-T2I) of decisive comparisons (Table 2).

    A cheap, plug-in improvement to frozen generators would be of practical value.

  2. Unusually candid analysis of what the guidance changes and where it fails.

    • App. H measures appearance against content with paired effect sizes (Table 15).
    • Apps. J and K report per-prompt regression rates, a blinded prompt-match judge (Tables 18–19), a systematic "inserted clouds" signature, and the weaker behavior in pixel space.

    This makes the method interpretable and checkable; much of my critique below rests on these appendices.

Critical weaknesses
  1. No baseline and no cost accounting.

    • Only vanilla is compared. Every result compares ManiGuide with the unguided sampler or with its own variants.
    • Best-of-N is missing. The paper motivates itself by the cost of best-of-N cherry-picking ("costly and wasteful", Sec. 1), yet reports no best-of-N comparison at matched compute.
    • No cost figures. There is no runtime, memory or FLOP figure; App. B gives only "8× NVIDIA A6000 GPUs". Yet each guided step decodes the clean estimate, re-encodes it and backpropagates through both (Alg. 2; Sec. 5, Q2), and Phase 1 generates a 3,000–4,000-image pool per model.
    • No other alternatives. There is no reward-gradient guidance with the same scorers, although all four are differentiable. There is also no comparison with MMD guidance (Mahdizadeh Sani et al.), which the paper calls "Closest to ours" (Sec. 2). It was presented at ICML 2026 on July 9, 2026, so it is not contemporaneous under ICLR's rule.

    This matters because ImageReward regresses on 40% (SD3.5) and 46% (PixNerd) of prompts (Apps. J–K). Best-of-N selection under the same scorer never scores below the vanilla sample it contains. Without these comparisons, I cannot judge whether the method is worth its cost.

  2. The evidence does not establish metric-specific "preference alignment" from self-distillation; a generic texture push fits it at least as well.

    • The target metric does not steer. If the metric-ranked reference drove the gains, guiding with metric M's reference should raise M the most. In Table 3:
      • that holds strictly in only 3 of 12 columns;
      • the ImageReward reference is best or tied-best in 10 of 12;
      • the target metric is the lowest in its own column three times. For example, SD3.5 PickScore gains +0.104 with the PickScore reference vs +0.136 with the ImageReward reference, and FLUX HPS gains +0.000.
    • Provenance barely matters. Real LAION images do about as well as self-distilled samples (Table 5: +0.085 vs +0.090 ImageReward), and wrong-style references still work (Table 17).
    • What changes reliably is texture. Sharpness rises +61.5% in 97.8% of pairs and total variation +42.2% in 99.2% (Table 15). Yet only about 60% of prompts improve on the targeted metric (App. J).
    • Ranking across captions mixes content with quality. The pool is ranked across different captions, so content is confounded with quality, which fits the inserted clouds: guided is named cloudier in 76% of sky pairs vs 16% for vanilla (App. J).
    • The stated principle hurts when applied as stated. Full distribution matching (ρ = 1) gives ImageReward −0.096 and PickScore −0.603 (Table 8), and only a heuristic 25% patch mask works.
    • The statistics are thin.
      • The defaults were selected on the 800 HPS v2.1 photo prompts, which are part of the Table 3 test set. ρ was chosen because "only ρ=0.25 keeps all four metrics positive".
      • No selection rule is given for the per-model and per-subject η and ρ.
      • No confidence intervals are reported, though many cells are +0.000 to +0.001.
      • Fig. 4a, captioned "flat", moves from about 0.04 (2k pool) to about 0.09 (3k, the default) and back to about 0.06 near 7k. That swing is comparable to the headline effect.
    • The ImageNet result is a different method. Table 1 labels it "+ ManiGuide", but it uses a real-image reference, DINOv3 features and mean + covariance matching "with no patch selection and no sliced transport" (App. F). It is evaluated only by FID, which is known to respond to aligning generated and real feature statistics (Kynkäänniemi et al., ICLR 2023). Its vanilla FIDs are also above the officially reported values: JiT-B/16 3.748 vs 3.66, and JiT-H/16 1.944 vs 1.86.

    This matters because the abstract claims to push a model "toward its own upper-bound quality as measured by any given metric". It also claims the method "improves or preserves every metric". On this evidence those claims are not supported, and a simpler explanation is not ruled out.

Questions for authors
  1. (Suggestion.)

    • How does ManiGuide compare, at matched wall-clock time, with best-of-N (N = 2 and 4) selected by the same scorer?
    • How does it compare with reward-gradient guidance using the same scorers, and with MMD guidance using the same 16-image reference?
    • Please also report per-image runtime and peak memory for Phase 2, and the cost of Phase 1.
  2. (Suggestion.) Please add these controls:

    • an unranked random-K reference and the worst-K set as the target;
    • real-image references for all three models;
    • a post-hoc sharpening baseline (e.g., an unsharp mask) matched in LPIPS or sharpness change;
    • within-prompt ranking (several samples per caption), to separate quality from content;
    • a ρ ablation at fixed total step norm. Eq. (8) scales the step by √ρ, so the total update at ρ = 1 is twice that at ρ = 0.25, while the update per edited location is about the same.

    These would show whether the ranked self-distilled reference matters beyond generic sharpening.

  3. (Clarification / suggestion.) Can you report per-prompt bootstrap confidence intervals and win rates for Table 3? The paired data already exist. Please re-select η, ρ and the pool size on prompts disjoint from HPS v2.1, and give the number of ties in the human study.

  4. (Suggestion.)

    • DreamBooth. Please report subject fidelity (DINO, CLIP-I) and prompt fidelity (CLIP-T) per subject. Also give the prompt and sample counts, and say how η was chosen for each subject. In Fig. 5 ("dog 5", rightmost pair), the guided image appears to render the red-brown dog black-and-white.
    • ImageNet. Please run the actual ManiGuide algorithm (self-distilled reference, sliced loss, patch mask) from the official vanilla settings, and report FD-DINOv2 and precision/recall alongside FID.

What would move me to 3 (weak accept):

  • compute-matched baselines that show a favorable cost/benefit trade-off (Q1);
  • controls showing the ranked self-distilled reference matters beyond generic sharpening (Q2);
  • claims rewritten to match the evidence: "improves the mean"; "any given metric", "upper-bound quality" and the "+ ManiGuide" label on the ImageNet pilot either removed or qualified.

Minor (no effect on my rating):

  • Fig. 4's caption "Reference-side scaling is flat" does not describe the synthetic-pool curve.
  • The human study (App. G) excludes ties without reporting how many there were. Each per-style cell rests on 10 prompts; the PixNerd photo cell is 54.8%.
  • Related work.
    • SW-Guidance (Lobashev et al., NeurIPS 2025) already puts a sliced-Wasserstein loss inside diffusion sampling, though for color distributions.
    • Training-based preference alignment is not discussed despite the title. Diffusion-DPO, for example, is cited only for the human-study protocol.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 14WX (1/4)

We thank the reviewer for the constructive feedback. We address each point below.

Compute-matched baselines (W1.1, W1.2, Q1.1, M1)

Every result compares ManiGuide with the unguided sampler or with its own variants

How does ManiGuide compare, at matched wall-clock time, with best-of-N (N = 2 and 4) selected by the same scorer?

At matched compute, ManiGuide outperforms DPS, DTS [2], VASR [1], and FKD on the objective. See Common concern 1 for the full comparison and Common concern 2 for best-of-2 and best-of-4.

Cost (W1.3, Q1.4)

There is no runtime, memory or FLOP figure

Please also report per-image runtime and peak memory for Phase 2, and the cost of Phase 1.

See Common concern 3 for per-image runtime, peak memory, FLOPs, generator evaluations, and the cost and amortization of Phase 1 (reference building, done once offline). ManiGuide takes 1.40x the vanilla sampling time and 1.15x its FLOPs, with the same 40 transformer calls and peak memory.

Reward-gradient guidance with the same scorers, and MMD guidance (W1.4, Q1.2, Q1.3)

How does it compare with reward-gradient guidance using the same scorers, and with MMD guidance using the same 16-image reference?

DPS with the same scorers. We compare ManiGuide with DPS, a reward-gradient guidance baseline, on the 800 HPS v2.1 photo prompts. DPS and ManiGuide give similar gains on the target metric for ImageReward and Aesthetic, but ManiGuide outperforms DPS on the other three metrics for every objective.

ObjectiveMethodImageRewardAestheticHPS v2.1PickScore
ImageRewardManiGuide+0.090+0.096+0.0090+0.042
ImageRewardDPS+0.073-0.007+0.0005+0.001
AestheticManiGuide+0.075+0.105+0.0066+0.031
AestheticDPS-0.003+0.125+0.0001+0.001
HPS v2.1ManiGuide+0.090+0.089+0.0074+0.019
HPS v2.1DPS+0.002+0.001+0.0005+0.007
PickScoreManiGuide+0.081+0.084+0.0068+0.011
PickScoreDPS+0.012+0.007+0.0026+0.251

MMD guidance. ManiGuide outperforms MMD guidance on all four metrics on the same SD3.5-M photo benchmark, using the same 16-image ImageReward-ranked reference.

MethodImageRewardAestheticHPS v2.1PickScore
MMD guidance-0.010+0.008-0.0025-0.040
ManiGuide+0.090+0.096+0.0090+0.042

We added both comparisons to App. L.1.

Per-prompt regressions (W1.5)

ImageReward regresses on 40% (SD3.5) and 46% (PixNerd) of prompts (Apps. J-K). Best-of-N selection under the same scorer never scores below the vanilla sample it contains

On SD3.5-M, we compare both ManiGuide and a second vanilla draw against the original vanilla sample for each prompt. ManiGuide scores lower on 40% of prompts (App. J), compared with 48.0% for vanilla resampling. On the photo prompts, drops greater than 0.5 ImageReward points are also less frequent with ManiGuide (3.5% vs. 11.1%).

To avoid regressions in ImageReward, we could keep the higher-scoring image from the original vanilla sample and one ManiGuide sample. This raises the mean ImageReward gain from +0.090 to +0.165. Selecting the better of two ManiGuide samples gives a larger mean gain of +0.269. We added this analysis to App. N.2.

The target metric does not steer (W2.1)

The target metric does not steer. If the metric-ranked reference drove the gains, guiding with metric M's reference should raise M the most

We agree that the paper overstates metric-specific steering. The more consistent finding is that ManiGuide improves or preserves mean preference scores across metrics and models.

To understand why a scorer's own reference doesn't always give it the largest gain, we compared how the scorers ranked the same 3,000 SD3.5-M samples. Spearman correlations ranged from 0.32 to 0.67, so the scorers often favor some of the same images. Such an overlap helps explain why ranking the reference pool with one scorer can also improve scores under another. We added this to App. N.4, and revised the claims (see The claims).

Provenance barely matters (W2.2)

Real LAION images do about as well as self-distilled samples (Table 5: +0.085 vs +0.090 ImageReward), and wrong-style references still work (Table 17)

We see this robustness as an advantage: ManiGuide does not require curated real images or a style-matched reference. Real LAION images give +0.085 ImageReward against +0.090 for self-distilled samples (Table 8), all 16 pairings of reference and target styles improve the target (Table 20), and self-distillation also works for DreamBooth subjects (Table 4), where matching real images are unavailable.

AuthorsOpenReview ↗

Response to Reviewer 14WX (2/4)

Sharpening baseline (W2.3, Q2, M2)

What changes reliably is texture. Sharpness rises +61.5% in 97.8% of pairs and total variation +42.2% in 99.2% (Table 15)

a post-hoc sharpening baseline (e.g., an unsharp mask) matched in LPIPS or sharpness change

To test how much sharpening explains ManiGuide's gains, we applied unsharp masking to vanilla images and matched ManiGuide's increase in sharpness. With ImageReward as the objective, ManiGuide outperforms unsharp masking on the objective.

MethodImageRewardAestheticHPS v2.1PickScore
ManiGuide+0.090+0.096+0.0090+0.042
Unsharp masking+0.038+0.093+0.0028+0.041

We added this baseline to App. L.2.

Random and worst-ranked references (Q2, M2)

an unranked random-K reference and the worst-K set as the target

We replaced the top-ranked 16 reference images with 16 randomly selected or 16 worst-ranked images. Both the worst-ranked reference and random reference gave smaller gains on ImageReward.

ReferenceImageRewardAestheticHPS v2.1PickScore
Top-ranked 16+0.090+0.096+0.0090+0.042
Random 16+0.084+0.089+0.0092+0.017
Worst-ranked 16+0.051+0.085+0.0038+0.003

We added this control to App. M.1.

Guided patch fraction at a fixed step size (W2.5, Q2)

a ρ\rho ablation at fixed total step norm. Eq. (8) scales the step by ρ\sqrt{\rho}, so the total update at ρ=1\rho = 1 is twice that at ρ=0.25\rho = 0.25

At the same total step norm, guiding all patches (ρ=1\rho=1) nearly matches the ImageReward gain from guiding 25% of patches, but PickScore drops.

Guided fractionImageRewardAestheticHPS v2.1PickScore
ρ=0.25\rho=0.25+0.090+0.096+0.0090+0.042
ρ=1\rho=1, same step size as ρ=0.25\rho=0.25+0.089+0.124+0.0137-0.059

We added this control to App. M.2.

Patch selection (W2.5)

Full distribution matching (ρ=1\rho = 1) gives ImageReward -0.096 and PickScore -0.603 (Table 8), and only a heuristic 25% patch mask works

We reversed the JS ranking to guide the 25% of patches that already match the reference best. The ImageReward gain fell from +0.090 to +0.030. Guiding a random 25% gives a similar gain to the default (+0.092 vs. +0.090). JS selection keeps guidance away from the best-matched patches and adds only about 0.03 s per image. We added both controls to App. M.3.

Real-image references for all three models (Q2)

real-image references for all three models

For the real-image reference, we used the real images paired with the same 3000 LAION captions as the self-distilled pool. We ranked them by ImageReward and built the reference from the 16 highest- and 16 lowest-ranked images. Self-distilled references produce higher objective gain than real-image references on all three models.

ModelReferenceImageRewardAestheticHPS v2.1PickScore
SD3.5-MSelf-distilled+0.090+0.096+0.009+0.042
Real images+0.085+0.088+0.009+0.026
PixNerd-T2ISelf-distilled+0.033+0.044+0.0022+0.063
Real images+0.021+0.043+0.0023+0.058
FLUX.1-devSelf-distilled+0.058+0.065+0.0058+0.023
Real images+0.045+0.045+0.0074-0.042

We added this comparison to App. M.4.

Cross-caption ranking and the inserted clouds (W2.4, Q2)

The pool is ranked across different captions, so content is confounded with quality, which fits the inserted clouds

within-prompt ranking (several samples per caption), to separate quality from content

We agree that ranking images across captions can mix content with quality. In a new pool of 3,000 images with four samples per caption, differences between captions accounted for 83% of the ImageReward variance. To test the effect of ranking within captions, we selected the 16 captions with the largest within-caption score spread. For each caption, we put its highest-scoring image in the top reference set and its lowest-scoring image in the bottom set, so both sets covered the same captions.

With captions matched, the ImageReward gain was only 0.001 lower than with the original reference. Among pairs with visible sky, Qwen3-VL-32B judged the guided image cloudier than the vanilla image in 81% of pairs, compared with 76% using the original reference. The added clouds therefore persist even when images are ranked within captions, so cross-caption ranking alone does not explain the effect. We added this analysis to App. M.5.

AuthorsOpenReview ↗

Response to Reviewer 14WX (3/4)

Selection of η\eta and ρ\rho (W3.1, Q3.2)

The defaults were selected on the 800 HPS v2.1 photo prompts, which are part of the Table 3 test set

No selection rule is given for the per-model and per-subject η\eta and ρ\rho

Please re-select η\eta, ρ\rho and the pool size on prompts disjoint from HPS v2.1

See Common concern 4 for the selection rule. We selected the hyperparameters on held-out sets.

Confidence intervals and win rates (W3.2, Q3.1)

Can you report per-prompt bootstrap confidence intervals and win rates for Table 3?

We ran ManiGuide on SD3.5-Medium with three independent seeds for each objective in Table 3 (ImageReward, Aesthetic, HPS v2.1, and PickScore). For each cell, we report the mean change from vanilla and a 95% paired bootstrap confidence interval over the 3,200 prompts.

TargetImageRewardAestheticHPS v2.1PickScore
ImageReward+0.072 [+0.065, +0.080]+0.098 [+0.092, +0.103]+0.0102 [+0.0099, +0.0106]+0.138 [+0.128, +0.148]
Aesthetic+0.043 [+0.035, +0.050]+0.098 [+0.092, +0.103]+0.0065 [+0.0061, +0.0068]+0.112 [+0.102, +0.123]
HPS v2.1+0.062 [+0.055, +0.070]+0.096 [+0.091, +0.101]+0.0082 [+0.0079, +0.0085]+0.103 [+0.093, +0.113]
PickScore+0.062 [+0.055, +0.070]+0.095 [+0.090, +0.100]+0.0080 [+0.0077, +0.0083]+0.114 [+0.104, +0.123]

For the same experiment, we report below how often the guided image outscored its vanilla counterpart.

TargetImageRewardAestheticHPS v2.1PickScore
ImageReward60.6%66.8%75.7%61.7%
Aesthetic56.0%66.7%66.6%58.2%
HPS v2.158.9%67.0%72.1%59.0%
PickScore59.4%67.1%71.2%60.1%

We added both tables to App. N.1.

Fig. 4a (W3.3, Minor)

Fig. 4's caption "Reference-side scaling is flat" does not describe the synthetic-pool curve.

We revised the caption to describe the curves: for the synthetic pool, the gain rises sharply from N = 2k to the default N = 3k and does not improve beyond it, while the real pool stays between +0.078 and +0.092.

Human study ties (Q3.3, Minor)

give the number of ties in the human study

Each model had 520 comparisons (13 raters x 40 prompts). For SD3.5-M, raters preferred the guided image 364 times, the vanilla image 110 times, and reported "no preference" 46 times (8.8%). For PixNerd-T2I, the counts were 211, 93, and 216 (41.5% ties). We added these counts to App. G.

The ImageNet pilot (W4.1)

Table 1 labels it "+ ManiGuide", but it uses a real-image reference, DINOv3 features and mean + covariance matching "with no patch selection and no sliced transport"

It is evaluated only by FID, which is known to respond to aligning generated and real feature statistics (Kynk"a"anniemi et al., ICLR 2023). Its vanilla FIDs are also above the officially reported values

We agree that the ImageNet pilot does not evaluate the full ManiGuide algorithm. We relabeled the Table 1 row "+ feature-moment guidance (pilot)" and clarified in Sec. 4.1 that the pilot motivates ManiGuide but does not validate it.

For the vanilla JiT baselines, we used the official checkpoint, settings, and evaluation protocol. Our FIDs are 0.08-0.09 higher than the published values. Hardware or numerical differences may account for this gap. We added this note to App. F.

The full ManiGuide method on ImageNet (Q4.2)

Please run the actual ManiGuide algorithm (self-distilled reference, sliced loss, patch mask) from the official vanilla settings, and report FD-DINOv2 and precision/recall alongside FID

On ImageNet with the official PixNerd-XL/P16 checkpoint and sampling settings, ManiGuide lowers FID from 2.107 to 2.056 and FD-DINOv2 from 90.21 to 87.08. Precision increases from 0.810 to 0.814, and recall stays at 0.588. We added this result to App. F.1.

DreamBooth (Q4.1)

Please report subject fidelity (DINO, CLIP-I) and prompt fidelity (CLIP-T) per subject. Also give the prompt and sample counts, and say how η\eta was chosen for each subject

In Fig. 5 ("dog 5", rightmost pair), the guided image appears to render the red-brown dog black-and-white

For each DreamBooth subject, we selected η\eta on 200 held-out LAION prompts using the rule in Common concern 4. We then used Qwen2.5-7B-Instruct [3] to adapt 500 Pick-a-Pic test prompts to each subject and generated one vanilla and one guided image per prompt. The table shows changes from vanilla in subject fidelity (mean CLIP-I and DINO cosine similarity to the subject's training images) and prompt alignment (CLIP-T).

AuthorsOpenReview ↗

Response to Reviewer 14WX (4/4)

Metriccancatdog 5dog 6teapotcat 2dog 7robot toyMean
Δ\DeltaCLIP-I-0.0032-0.0102-0.0099-0.0043-0.0034-0.0066-0.0071-0.0076-0.0065
Δ\DeltaDINO-0.0105-0.0092-0.0078-0.0072-0.0037-0.0089-0.0137-0.0025-0.0079
Δ\DeltaCLIP-T+0.0021+0.0019+0.0037+0.0020+0.0014+0.0005+0.0024-0.0008+0.0017

Subject fidelity decreases slightly for all eight subjects, with average drops of 0.8% in CLIP-I and 1.3% in DINO relative to vanilla, while prompt alignment is nearly unchanged. The rightmost dog 5 pair in Fig. 5 indeed shows a visible subject-fidelity failure. We flagged this example in the caption of Fig. 5, added the table to App. P, and added the selection rule to Sec. 4.3.

The claims (W4.2, M3)

the abstract claims to push a model "toward its own upper-bound quality as measured by any given metric". It also claims the method "improves or preserves every metric"

On this evidence those claims are not supported, and a simpler explanation is not ruled out

We agree that both claims are too strong. We revised the abstract to say that ManiGuide "raises mean preference scores for images generated by a frozen model" and that "in our main text-to-image experiments, none of the four evaluated preference metrics decreases on average." We made the same changes in the introduction and conclusion.

Related work (Minor)

SW-Guidance (Lobashev et al., NeurIPS 2025) already puts a sliced-Wasserstein loss inside diffusion sampling, though for color distributions

Training-based preference alignment is not discussed despite the title

We added SW-Guidance [4] to the related work, along with a paragraph on training-based preference alignment, including Diffusion-DPO, DRaFT [5], AlignProp [6], ReFL, and D3PO [7].

We hope this addresses the reviewer's concerns.

References:

[1] Shekhar et al. VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion. arXiv 2026.

[2] Jain et al. Diffusion Tree Sampling: Scalable Inference-Time Alignment of Diffusion Models. NeurIPS 2025.

[3] Yang et al. (Qwen Team). Qwen2.5 Technical Report. arXiv 2412.15115, 2024.

[4] Lobashev et al. Color Conditional Generation with Sliced Wasserstein Guidance. NeurIPS 2025.

[5] Clark et al. Directly Fine-Tuning Diffusion Models on Differentiable Rewards. ICLR 2024.

[6] Prabhudesai et al. Aligning Text-to-Image Diffusion Models with Reward Backpropagation. ICLR 2024.

[7] Yang et al. Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model. CVPR 2024.

Reviewer_XNYJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ManiGuide, a training-free inference-time alignment method for diffusion models. The method first generates a pool of images using the base model, ranks them according to a reward or preference metric, and constructs a reference distribution in deep-feature space from the highest-scoring samples. During inference, it identifies patches whose feature distributions deviate most from this reference and guides them toward it using sliced-Wasserstein optimal transport. Experiments demonstrate that ManiGuide consistently improves or preserves preference metrics across different reward models, generative architectures, and generation settings, including class-conditional and text-to-image generation.

Critical strengths

1.The method does not require any additional training or reward model evaluations during sampling, making it easy to apply to different preference objectives.

2.The authors evaluate the method across different generative models, preference metrics, and generation tasks, showing that it works well across different settings.

3.The paper includes detailed ablation studies on different components of the method, which help explain the importance of the design choices.

Critical weaknesses
  1. The paper does not compare ManiGuide with existing inference-time alignment methods such as FKD[1], VASR[2], DTS[3], and DPS[4], etc. . Comparing against these methods under matched inference-time compute budgets would help show whether ManiGuide offers meaningful improvements over existing approaches.

  2. While the method does not require training, it still involves generating and ranking a pool of images offline, as well as feature extraction and gradient computation during every denoising step. The authors should report the computational overhead and inference latency to give a clearer picture of how efficient the method is in practice.

  3. Although the authors study how reference pool size affects performance, they do not explore how the method scales with additional online compute. It would be useful to see an ablation varying the number of guided denoising steps and sliced-Wasserstein projections to understand the compute–performance tradeoff. The authors should also discuss how the offline cost is amortized when generating multiple images.

[1] Singhal, R., Horvitz, Z., Teehan, R., Ren, M., Yu, Z., McKeown, K., and Ranganath, R. "A General Framework for Inference-time Scaling and Steering of Diffusion Models." Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025. https://openreview.net/forum?id=Jp988ELppQ

[2] Shekhar, S., Mukherjee, S., Zhang, J. Y., and Zhang, T. "VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion." arXiv preprint arXiv:2604.06779, 2026. https://arxiv.org/abs/2604.06779

[3] Jain, V., Sareen, K., Pedramfar, M., and Ravanbakhsh, S. "Diffusion Tree Sampling: Scalable Inference-Time Alignment of Diffusion Models." Advances in Neural Information Processing Systems (NeurIPS), 2025. https://openreview.net/forum?id=3D88hCO0Gd

[4] Chung, H., Kim, J., McCann, M. T., Klasky, M. L., and Ye, J. C. "Diffusion Posterior Sampling for General Noisy Inverse Problems." International Conference on Learning Representations (ICLR), 2023. https://openreview.net/forum?id=OnD9zGAGT0k

Questions for authors

See weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer XNYJ

We thank the reviewer for the constructive feedback. We address each point below.

Compute-matched comparisons (W1)

The paper does not compare ManiGuide with existing inference-time alignment methods such as FKD[1], VASR[2], DTS[3], and DPS[4]

See Common concern 1.

Computational overhead and inference latency (W2)

The authors should report the computational overhead and inference latency

See Common concern 3.

Scaling with online compute, and amortization of the offline cost (W3)

an ablation varying the number of guided denoising steps and sliced-Wasserstein projections to understand the compute-performance tradeoff

The authors should also discuss how the offline cost is amortized when generating multiple images

Guiding more denoising steps generally improves performance on the objective, while increasing the number of sliced-Wasserstein rotation sets gives similar results. The table reports changes from vanilla on 800 HPS v2.1 photo prompts using SD3.5-Medium and ImageReward as the objective, with latency measured on one A6000.

Guided steps out of 40Rotation sets RRAdded time/imageImageRewardAestheticHPS v2.1PickScore
101+0.38 s+0.038+0.023+0.0028+0.037
108+0.58 s+0.051+0.034+0.0026+0.036
201+0.72 s+0.056+0.054+0.0051+0.060
208+1.13 s+0.060+0.055+0.0050+0.059
401+1.38 s+0.092+0.097+0.0088+0.029
40, default setting8+1.99 s+0.090+0.096+0.0090+0.042

We added this ablation to App. M.7. Also see Common concern 3 for amortization.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Summary (1/2)

We thank the reviewers for their constructive feedback. We are encouraged that reviewers found ManiGuide simple and practical, requiring no training or online metric computation (XNYJ, 14WX, StsD), effective across generators and preference metrics (XNYJ, 14WX, StsD), perceptually meaningful (14WX, StsD), and supported by well-motivated ablations (XNYJ, StsD). Unless noted otherwise, the new experiments use SD3.5-Medium on the 800 HPS v2.1 photo prompts, with ImageReward as the objective. Changes in the revised paper are in blue, and all numbers below refer to it.

  • Matched-compute comparisons with DPS, DTS, VASR, and FKD, including composition with FKD (Common concern 1).
  • Best-of-NN comparisons using vanilla and ManiGuide candidates (Common concern 2).
  • Latency, FLOPs, peak memory, transformer evaluations, and offline reference amortization (Common concern 3).
  • Held-out selection of η\eta and ρ\rho (Common concern 4).
  • Scaling with guided steps and sliced-Wasserstein rotations (XNYJ).
  • DPS comparisons for all four objectives and MMD guidance with the same reference (14WX).
  • Controls for reference ranking and source, matched sharpening, patch fraction and selection, and within-caption ranking (14WX).
  • A PCA-only control for JS-based direction selection (StsD).
  • Constraint-level prompt faithfulness and the distribution of per-prompt reward changes (StsD).
  • Per-subject DreamBooth fidelity and prompt alignment (14WX, StsD).
  • Three-seed paired bootstrap confidence intervals for the SD3.5-Medium results in Table 3 (14WX, StsD).
  • Per-prompt win rates for the same SD3.5-Medium results (14WX).
  • Human-study tie counts for both models (14WX).
  • Full ManiGuide evaluation on ImageNet, including FID, FD-DINOv2, precision, and recall (14WX).

Common concern 1: comparisons with inference-time alignment methods at matched compute (Reviewers XNYJ, 14WX, StsD)

We compare ManiGuide with DPS, DTS [2], VASR [1], and FKD at matched compute on 800 HPS v2.1 photo prompts using SD3.5-Medium and ImageReward as the objective. ManiGuide gives the largest gains in the objective. As an ad hoc guidance method, ManiGuide can also be composed with FKD.

MethodImageRewardAestheticHPS v2.1PickScore
DPS+0.073-0.007+0.0005+0.001
DTS+0.037+0.032+0.0017+0.042
VASR+0.051+0.027+0.0016+0.042
FKD+0.050+0.031+0.0019+0.047
ManiGuide+0.090+0.096+0.0090+0.042
ManiGuide composed with FKD+0.119+0.118+0.0089+0.076

Common concern 2: best-of-N comparison (Reviewers 14WX, StsD)

We compare ManiGuide with best-of-N on SD3.5-M, using ImageReward as the objective and the 800 HPS v2.1 photo prompts. We also test their combination, where best-of-N selects from ManiGuide-generated candidates. ManiGuide-generated candidates add +0.068 ImageReward to best-of-2 and +0.063 to best-of-4.

Methods/img (A6000)ImageRewardAestheticHPS v2.1PickScore
ManiGuide6.96 (1.40x)+0.090+0.096+0.0090+0.042
Best-of-210.0 (2.0x)+0.201+0.013+0.0034+0.112
ManiGuide + Best-of-214.0 (2.8x)+0.269+0.123+0.0124+0.111
Best-of-420.1 (4.0x)+0.316+0.035+0.0055+0.140
ManiGuide + Best-of-428.0 (5.6x)+0.379+0.143+0.0142+0.166

Common concern 3: inference and offline cost (Reviewers XNYJ, 14WX, StsD)

Inference cost. Compared with vanilla, ManiGuide adds 40% wall-clock time (6.96 vs. 4.97 s/image) and 15% FLOPs, with no increase in peak memory or transformer calls. The table reports per-image costs for SD3.5-M on one RTX A6000 (512 px, 40 steps, batch size 1), excluding the one-time reference build. Best-of-NN peak memory includes ImageReward.

Methods/imagePeak memoryTransformer callsTFLOPs/image
Vanilla4.9717.6 GB40173.5
ManiGuide6.9617.6 GB40200.0
Best-of-210.019.4 GB80347.3
Best-of-420.119.4 GB160694.6

Offline cost. ManiGuide builds one reusable reference per model and metric (0.55 hours on 8 A6000s, or 4.4 GPU-hours for 3,000 images scored with ImageReward). Including this build, the total overhead over vanilla is 17.7, 3.6, and 2.1 s/image at 1k, 10k, and 100k generations, respectively.

AuthorsOpenReview ↗

General Summary (2/2)

Common concern 4: hyperparameter selection (Reviewers 14WX, StsD)

For the HPS v2.1 experiments, we selected the hyperparameters on 400 LAION prompts disjoint from the benchmark. For DreamBooth, we selected the hyperparameters separately for each subject on 200 held-out LAION prompts per subject. In both cases, we chose the setting with the largest mean gain on the target metric among those that matched or exceeded vanilla on all four preference metrics.


[1] Shekhar et al. VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion. arXiv 2026.

[2] Jain et al. Diffusion Tree Sampling: Scalable Inference-Time Alignment of Diffusion Models. NeurIPS 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.