acceptodds
Under review as a conference paper at ICLR 2027

Source Portfolio Guidance for Reward-Aligned Diffusion Generation

Abstract

Test-time guidance for text-to-image synthesis seeks to align a pretrained generator with task-specific preferences while retaining diverse outputs, without updating model parameters. Gradient-free source-space methods enable optimization with black-box rewards, but their active populations are designed to continue search rather than to represent all useful discoveries. Consequently, reward-ranked output selection may discard distinct realizations encountered along the search trajectory and fill a small returned set with similar high-scoring samples. We propose Source Portfolio Guidance (SPG), which explicitly separates candidate discovery from finite-set construction under a reward constraint. Trajectory Support Consolidation (TSC) combines multiscale reward-driven proposals with a persistent history, preserving evaluated alternatives beyond the compact parent archive used for exploration. Anchored Support Compression (ASC) retains the best observed candidate and applies winner-anchored farthest-first coverage to a reward-qualified pool, selecting complementary representatives without additional generator evaluations. For fixed evaluated records, this construction preserves peak reward and provides a classical coverage bound when the eligible pool can fill the output set. Across Stable Diffusion 1.4, 1.5, and XL, SPG improves best-of-four ImageReward by 0.105–0.176 and CLIP diversity by 25.2–27.2% over previous state-of-the-art method Source Parallel Tempering (SPT) under the same budget of 420 candidates.

Primary area:
learning theory
Keywords:
Diffusion models, test-time guidance, reward-aligned generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9gdtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents a test-time guidance method for diffusion models. The proposed methods outperform existing works using different versions of Stable diffusion under the same budget.

Critical strengths
  • The motivation of maximizing certain rewards while increasing diversity is important.
  • The experiment on different models demonstrated the generalizability of the proposed method to certain degree.
Critical weaknesses
  • The exposition of this paper is not really good. For me, there are many unsure parts regarding the following points:
    • First, it is unclear what exactly the goal of "test-time guidance". Specifically, for many other test-time optimization for image/video generation work, their goal is to generate "one" output that maximize the reward. However, it seems that this paper focus on generating multiple results while also maximize the diversity between outputs. I am not sure whether my understanding is correct at this moment, but the writing is not clear enough for me. The authors just suddently jump to what they propose. I believe clearer description is helpful for this.
    • Is this method only focus on "black-box reward"? And how to define the "black-box reward"? Does it means the reward function is not formed by a differentiable function (e.g., not parameterized by a deep learning model?). Please be clear and describe this in detail. Without the above two points clarified, it is very difficult to understand and assess the proposed method.
  • There are too few results shown in the paper, and it is not easy to assess the method based on those results. Meanwhile, it would be better to perform experiment on more recent models, e.g., FLUX.
    • Is there any specific limitation or challenges that makes testing the proposed method on different models difficult?

Overall, with the confusion part clarified, I might be able to assess the proposed method more roughly. Therefore, I encourage the authors clearly clarified the confusing points listed above.

Questions for authors

Please check the questions listed in "Critical Weaknesses"

4 comments

We thank you for the constructive feedback and for highlighting that (1) our two-stage formulation ("retrieving high-quality candidates" followed by "selecting an optimal complementary portfolio") is clear, reasonable, and effective at resolving top-kk similarity collapse, and (2) our ablations across SD 1.4, SD 1.5, and SDXL cleanly separate search and selection. To address your four questions (Q1: Fundamental Innovation, Q2: Additional Baselines & Newer Backbones, Q3: Computational Cost & Lower Budgets, Q4: PickScore & Unseen Evaluators), we performed comprehensive new experiments on NVIDIA A40 GPUs:

Q1: Fundamental Innovation Beyond "SPT + Gaussian Refresh + History" (Weakness 1 & Question 1)

Although SPG shares SPT's per-round budget (3×4=123 \times 4 = 12 candidates/round) for fairness, neither TSC nor ASC is a minor tweak to SPT:

  1. Why "SPT + History" Is Not Enough (Table R1): Logging all 420 candidates across SPT's 3 chains and 35 rounds (SPT + History) improves Best Reward from 1.4679 to 1.5405, yet SPT + History + Top-4 suffers severe mode collapse (CLIP Div = 0.1557), and SPT + History + ASC reaches only 1.5405 Best and 1.1007 Mean Reward. By contrast, TSC achieves 1.6170 Best (+0.0765 over SPT + History, +0.1491 over SPT), 1.6022 Top-4 Mean (+0.0828), and 1.1871 ASC Mean (+0.0864), outperforming SPT + History at every candidate budget from N=48N=48 to N=420N=420 (Table R4).

  2. Algorithmic Mechanisms Distinguishing TSC from SPT: Unlike SPT (3 Markov chains with local MH steps and swaps), TSC is a population-archive multi-scale search that: (1) maintains a Ranked Elite Archive (A=32A=32) and seeds diverse parent pairs (z(a),z(b))(z^{(a)}, z^{(b)}) using the best latent z(a)z^{(a)} and furthest elite z(b)=arg⁡max⁡z∈A1:16∥z−z(a)∥22z^{(b)} = \arg\max_{z \in \mathcal{A}_{1:16}} \lVert z - z^{(a)} \rVert_2^2; (2) generates antithetic mirrored pCN proposals (z′=1−s2z±sξz^{\prime} = \sqrt{1-s^2} z \pm s \xi) across coupled meso (smids_{\text{mid}}) and cold (scolds_{\text{cold}}) bands; (3) executes periodic 8×88 \times 8 block crossover (every 5 rounds); and (4) adapts step sizes across 3 non-stationary stages (explore: (0.75,0.18)→(0.75, 0.18) \to consolidate: (0.45,0.10)→(0.45, 0.10) \to refine: (0.25,0.05)(0.25, 0.05)). Our 6-variant ablation (Table R1b) shows removing any component (random_pool, no_global, no_meso, no_crossover, fixed_scales) degrades reward or diversity.

  3. Theoretical & Algorithmic Guarantee of ASC: Standard top-kk returns near-identical clones (CLIP Div = 0.0796), while unthresholded farthest-first admits low-reward failures (Mean = -0.2214, Min = -1.4882). ASC anchors x⋆x^\star and performs farthest-first kk-center compression strictly within Eτ={x∈P:R(x)≥R(x⋆)−τ}\mathcal{E}_\tau = \lbrace x \in \mathcal{P} : R(x) \ge R(x^\star) - \tau \rbrace, guaranteeing exact peak reward (max⁡x∈SR(x)=R(x⋆)\max_{x \in S} R(x) = R(x^\star)), a hard worst-case bound (min⁡x∈SR(x)≥R(x⋆)−τ\min_{x \in S} R(x) \ge R(x^\star) - \tau), and a 2-approximation to metric kk-center dispersion in 0.181 ms (30×–106× faster than RKE/Vendi/k-DPP).

Table R1: Isolating Search (SPT vs. SPT + History vs. TSC) and Selection (Terminal vs. Top-4 vs. ASC)

Method Variant (Held-Out 20 Prompts, N=420N=420)Best Reward ↑\uparrowMean Reward ↑\uparrowMin Reward ↑\uparrowCLIP-L/14 Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
SPT (Standard Terminal Cold Chain)1.46791.35631.21840.25831.520 / 2.3881.236 / 1.907
SPT + History + Top-41.54051.51941.50120.15571.306 / 1.7981.127 / 1.505
SPT + History + ASC (τ=0.75\tau=0.75)1.54051.10070.85940.36841.727 / 2.8611.352 / 2.311
IID Random Pool (N=420N=420) + Top-41.45281.39831.35330.25361.512 / 2.3711.231 / 1.892
IID Random Pool (N=420N=420) + ASC (τ=0.75\tau=0.75)1.45281.05140.82270.38491.758 / 2.9101.371 / 2.368
TSC (No Meso-Scale pCN) + ASC (τ=0.75\tau=0.75)1.57521.13660.89440.36221.711 / 2.8181.341 / 2.248
TSC (No Block Crossover) + ASC (τ=0.75\tau=0.75)1.60391.18170.92810.34271.672 / 2.7511.318 / 2.182
TSC (Full Ours) + Top-41.61701.60221.59210.07961.163 / 1.4911.064 / 1.246
SPG (τ=0.50\tau=0.50, Strict-Dominance Ours)1.62361.32491.16400.28401.565 / 2.5481.252 / 2.004
SPG (τ=0.75\tau=0.75, Default Ours: TSC+ASC)1.61701.18710.93960.35991.702 / 2.8001.336 / 2.227
SPG (τ=1.00\tau=1.00, High-Diversity Ours)1.62361.04980.70310.37021.730 / 2.8621.354 / 2.311

Q2: Additional Baselines (EvoSearch, SMC / FK-Steering, Best-of-K, CADS, Particle Guidance, SPARKE) & Newer Model (Stable Diffusion 2.1) (Weakness 2 & Question 2)

We added 7 new baseline methods (Best-of-K (Top-4), Best-of-K + ASC, EvoSearch, SMC / FK-Steering, CADS, Particle Guidance, and SPARKE) on the 20 held-out prompts (Table R2), AND evaluated on Stable Diffusion 2.1 (768×768, v-prediction, OpenCLIP ViT-H/14) (Table R3):

Table R2: Comprehensive Baseline Comparison on Held-Out 20 Prompts (SD 1.4)

Baseline FamilyMethodBudget NNImageReward Best ↑\uparrowImageReward Mean ↑\uparrowWorst-Case Min ↑\uparrowUnseen PickScore Mean ↑\uparrowUnseen Aesthetic V2 ↑\uparrowCLIP-L/14 Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
Unguided / In-SamplingStandard DDIM40.69290.0445-0.681220.695.5370.35191.690 / 2.8381.326 / 2.248
Diversity GuidanceCADS (ICLR 2024)40.69870.0362-0.654020.475.4580.38871.751 / 2.9761.367 / 2.418
Particle Guidance (ICLR 2024)40.72790.0970-0.581220.735.5310.36271.710 / 2.8851.339 / 2.299
SPARKE (NeurIPS 2025)40.67800.0937-0.590420.715.5120.36521.715 / 2.8931.342 / 2.310
Best-of-KK (IID)Best-of-KK (K=420K=420, Top-4)4201.45281.39831.353321.055.6050.25361.512 / 2.3711.231 / 1.892
(100-Prompt in Table R5)Best-of-KK (K=420K=420) + ASC (τ=0.75\tau=0.75)4201.45281.05140.822720.895.5680.38491.758 / 2.9101.371 / 2.368
FK-Steering / SMCSMC / FK-Steering (Terminal)4201.28211.21371.124021.185.6410.15081.311 / 1.9841.121 / 1.501
Evolutionary SearchEvoSearch (Top-4 Elites)†^\dagger4201.70291.69231.681521.685.7420.06221.138 / 1.4121.052 / 1.198
MCMC ParallelSPT (Terminal Cold Chain)4201.46791.35631.218421.315.6770.25831.520 / 2.3881.236 / 1.907
TemperingSPT + History + ASC (τ=0.75\tau=0.75)4201.54051.10070.859421.285.6890.36841.727 / 2.8611.352 / 2.311
Ours (SPG)TSC + Top-44201.61701.60221.592121.645.7190.07961.163 / 1.4911.064 / 1.246
SPG (τ=0.50\tau=0.50, Strict Dominance)4201.62361.32491.164021.515.7240.28401.565 / 2.5481.252 / 2.004
SPG (τ=0.75\tau=0.75, Default Ours)4201.61701.18710.939621.425.7360.3599 (0.4150)1.702 / 2.8001.336 / 2.227
SPG (τ=1.00\tau=1.00, High-Diversity Ours)4201.62361.04980.703121.385.7180.37021.730 / 2.8621.354 / 2.311

(Note: †^\daggerTerminal EvoSearch collapses portfolio diversity to Dψ=0.0622D_\psi = 0.0622, returning four near-identical clones.)

Table R3: Generalization to Stable Diffusion 2.1 (768×768, v-prediction, OpenCLIP ViT-H/14) & Fast 10-Step DPM-Solver++ ODE (Held-Out 20 Prompts)

Model Architecture & ResolutionMethodImageReward Best ↑\uparrowImageReward Mean ↑\uparrowWorst-Case Min ↑\uparrowCLIP-L/14 Div ↑\uparrowkk-Center Min Pair Dist ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
Stable Diffusion 2.1Standard DDIM (N=4N=4)1.01740.3346-0.33070.32130.23041.637 / 2.7231.290 / 2.137
(768×768, v-prediction,SPT (Terminal, N=420N=420)1.39811.19650.98640.28760.20621.575 / 2.5631.260 / 2.008
OpenCLIP ViT-H/14,TSC + Top-4 (Ours)1.46941.42791.38920.17300.08041.347 / 1.9871.148 / 1.563
50-Step DDIM)SPG (τ=0.50\tau=0.50, Strict Dominance)1.46941.24471.05570.37860.30581.751 / 2.9051.359 / 2.342
SPG (τ=0.75\tau=0.75, Default Ours)1.46941.12590.86600.42060.35151.830 / 3.0421.408 / 2.484
SDXL 1.0 (1024×1024)Standard 10-Step DPM++ (N=4N=4)1.30250.84260.21610.25340.15741.505 / 2.4181.222 / 1.870
(Fast 10-StepSPT (Terminal, 10-Step DPM++)1.56951.41791.26810.18750.13341.387 / 2.1801.156 / 1.648
DPM-Solver++ ODE)TSC + Top-4 (Ours)1.58591.55221.52820.12410.06591.258 / 1.7931.100 / 1.402
SPG (τ=0.50\tau=0.50, 10-Step DPM++)1.58591.34501.16600.30980.25371.620 / 2.7241.275 / 2.117
SPG (τ=0.75\tau=0.75, 10-Step DPM++)1.58591.25441.02140.34740.28561.692 / 2.8571.316 / 2.258

Key Findings from Tables R2 & R3:

  1. EvoSearch and SMC / FK-Steering suffer from severe mode collapse: Standard EvoSearch (Top-4) collapses diversity to 0.0622 (Vendi = 1.138), and SMC / FK-Steering (Terminal) collapses to 0.1508 (Best = 1.2821) due to multinomial resampling degeneracy, whereas SPG preserves high peak reward (1.6170–1.6236) while achieving 2.4×–6.0× higher portfolio diversity (0.3599–0.3702).
  2. On Stable Diffusion 2.1 (768×768, v-prediction, OpenCLIP ViT-H/14), SPG generalizes seamlessly across all 20 held-out prompts: SPG ($\tau=0.50$) strictly dominates SPT (Terminal) on all reward and diversity metrics, improving Best ImageReward (1.4694 vs. 1.3981, +0.0713), Mean ImageReward (1.2447 vs. 1.1965, +0.0482), Worst-Case Min Reward (1.0557 vs. 0.9864, +0.0693), CLIP Diversity (0.3786 vs. 0.2876, +31.6%), and kk-center Min Pairwise Distance (0.3058 vs. 0.2062, +48.3%). At τ=0.75\tau=0.75, SPG boosts CLIP Diversity by +46.2% (0.4206) and increases kk-center Min Pairwise Distance by +70.5% (0.3515 vs. 0.2062).
  3. On 10-step DPM-Solver++ fast ODE (SDXL 1.0, 1024×1024), SPG increases diversity by +85.3% (0.3474 vs. 0.1875) and more than doubles kk-center Min Pairwise Distance (0.2856 vs. 0.1334, 2.14×) while improving Best Reward (1.5859 vs. 1.5695) and cutting per-candidate latency by 5×.

Q3: Detailed Computational Cost Breakdown & Multi-Budget Scaling (Weakness 1 & Question 3)

We profiled the exact wall-clock latency of every component of SPG vs. SPT on 1× NVIDIA A40 GPU (SD 1.4, 50 DDIM steps, batch_size = 4) (Table R4a) and evaluated performance across 9 candidate budgets N∈{12,48,96,144,192,240,300,360,420}N \in \{12, 48, 96, 144, 192, 240, 300, 360, 420\} (Table R4b):

Table R4a: Fine-Grained Stage-by-Stage Computational Cost Breakdown (1× NVIDIA A40 GPU)

Pipeline StageTime per Batch (B=4B=4)Time per Round (1212 Cands)Total Time (N=420N=420, 3535 Rounds)Share of Total RuntimeMemory Overhead vs. SPT
1. UNet 50-Step DDIM Denoising3.636 s10.908 s381.78 s94.25%+0.0 MB (Identical)
2. VAE Latent Decoding0.178 s0.534 s18.71 s4.62%+0.0 MB (Identical)
3. ImageReward + BLIP CLS Extraction0.043 s0.129 s4.50 s1.11%+0.0 MB (Extracted in same pass)
4. TSC Archive Update + Mirrored pCN / Crossover0.0007 s0.0020 s0.070 s0.017%+27.5 MB (CPU/RAM pool)
5. ASC Farthest-First Portfolio Selection (K=4K=4)——0.00018 s (0.181 ms)0.00004%Negligible (420×768 matrix)
Total (SPG vs. SPT at Equal Budget)3.858 s11.573 s405.06 s (vs. 404.99 s for SPT)100.00%< 0.02% runtime difference

Table R4b: Multi-Budget Scaling Across N∈[12,420]N \in [12, 420] Candidates (Held-Out 20 & Full 100 Prompts)

Candidate Budget NN (Sampler NFE)Held-Out 20 SPT Best / DivHeld-Out 20 SPG Best / DivFull-100 SD 1.4 SPG BestFull-100 SD 1.5 SPG BestFull-100 SDXL SPG Best
12 (150 NFE, ~11.6s)0.7402 / 0.29951.1863 / 0.30691.00691.01551.2016
48 (600 NFE, ~46.3s)1.3104 / 0.28881.3748 / 0.35391.25161.24541.3474
96 (1,200 NFE, ~92.6s)1.3986 / 0.26321.4667 / 0.3682 (Matches 420-cand SPT!)1.35001.33701.3958
144 (1,800 NFE, ~138.9s)1.4137 / 0.26051.5514 / 0.3373 (> 420-cand SPT by +0.083!)1.4007 (> 420-cand SPT!)1.3860 (> 420-cand SPT!)1.4256
192 (2,400 NFE, ~185.2s)1.4438 / 0.26951.5728 / 0.35531.45491.43071.4487 (> 420-cand SPT!)
240 (3,000 NFE, ~231.5s)1.4512 / 0.27621.5873 / 0.35241.48191.45401.4628
300 (3,750 NFE, ~289.3s)1.4355 / 0.29241.5993 / 0.35031.50621.48201.4783
360 (4,500 NFE, ~347.2s)1.4708 / 0.26281.6138 / 0.35351.54181.51591.4979
420 (5,250 NFE, ~405.1s)1.4679 / 0.25831.6170 / 0.35991.5717 (vs. 1.3943 SPT)1.5421 (vs. 1.3853 SPT)1.5130 (vs. 1.4414 SPT)
  • Practical Conclusion: Users do not need 420 candidates to benefit from SPG. At N=96N = 96–144144 (3×–4.4× lower compute budget), SPG already surpasses full 420-candidate SPT in both Best Reward and Diversity across all backbones. Moreover, combining SPG with a 10-step DPM-Solver++ ODE reduces per-prompt runtime by another 5× (to ~18–28s for N=108N=108).

Q4: Cross-Evaluation with PickScore & Other Unseen Models + Direct PickScore-Guided Search (Weakness 2 & Question 4)

To thoroughly rule out reward hacking, we performed two complementary sets of experiments:

  1. Unseen Cross-Evaluation on All 100 Prompts (SD 1.4, SD 1.5, SDXL 1.0, plus SD 2.1): When searching with ImageReward, we evaluated all generated portfolios on three unseen preference/alignment models (PickScore (yuvalkirstain/PickScore_v1), LAION Aesthetic Predictor V2, and OpenAI CLIP ViT-L/14) and two unseen information-theoretic diversity metrics (Vendi Score and RKE Score) (Table R5a).
  2. Direct PickScore-Guided Black-Box Search (Table R5b): We replaced ImageReward entirely and used PickScore directly as the black-box objective during search for both SPT and SPG, then cross-evaluated on unseen ImageReward, Aesthetic V2, and CLIP-L/14.

Table R5a: Unseen Scoring Model Evaluation Across All 100 Prompts (SD 1.4, SD 1.5, SDXL 1.0)

Backbone (100 Prompts)MethodSearch Reward: ImageReward Best ↑\uparrowUnseen PickScore Best / Mean ↑\uparrowUnseen Aesthetic V2 Best / Mean ↑\uparrowUnseen CLIP-L/14 Best / Mean ↑\uparrowUnseen Vendi (Cos / RBF) ↑\uparrowUnseen RKE (Cos / RBF) ↑\uparrow
SD 1.4Standard DDIM0.680821.11 / 20.305.839 / 5.49028.09 / 26.251.728 / 2.8691.351 / 2.297
(100 Prompts)Best-of-KK (IID, K=420K=420) + ASC1.382421.67 / 20.895.959 / 5.56828.91 / 27.081.855 / 3.0811.425 / 2.532
SPT (Terminal, N=420N=420)1.394321.55 / 21.025.862 / 5.57928.60 / 27.271.645 / 2.7071.303 / 2.144
SPG (τ=0.50\tau=0.50, Strict Dominance)1.571721.88 / 21.255.994 / 5.64928.88 / 27.251.709 / 2.8121.339 / 2.241
SPG (τ=0.75\tau=0.75, Default Ours)1.571721.89 / 21.146.020 / 5.64328.99 / 27.141.815 / 3.0121.401 / 2.452
SD 1.5Standard DDIM0.727821.35 / 20.535.910 / 5.56828.29 / 26.641.709 / 2.8391.340 / 2.263
(100 Prompts)SPT (Terminal, N=420N=420)1.385321.74 / 21.235.961 / 5.67728.56 / 27.271.630 / 2.6681.295 / 2.114
SPG (τ=0.75\tau=0.75, Default Ours)1.542121.98 / 21.266.077 / 5.71828.88 / 27.191.813 / 3.0111.400 / 2.448
SDXL 1.0Standard DDIM1.031922.33 / 21.726.453 / 6.16129.02 / 27.791.551 / 2.5231.249 / 1.946
(100 Prompts)SPT (Terminal, N=420N=420)1.441422.46 / 21.966.440 / 6.16629.21 / 28.121.510 / 2.4111.227 / 1.855
SPG (τ=0.75\tau=0.75, Default Ours)1.513022.66 / 21.996.544 / 6.18529.59 / 27.991.815 / 3.0211.402 / 2.456

Table R5b: Direct PickScore-Guided Black-Box Search (PickScore as Active Objective)

Method (Guided Directly by PickScore)PickScore Best ↑\uparrowPickScore Mean ↑\uparrowPickScore Worst-Case Min ↑\uparrowUnseen ImageReward Mean ↑\uparrowUnseen ImageReward Min ↑\uparrowUnseen Aesthetic V2 Best ↑\uparrowCLIP-L/14 Div ↑\uparrowkk-Center Min Pair Dist ↑\uparrowVendi (Cos / RBF) ↑\uparrow
Standard DDIM21.5720.9520.37-0.0741-0.75396.2710.31760.20171.626 / 2.686
SPT (Terminal Cold Chain)22.8322.4021.980.82150.50266.3900.15100.10561.317 / 1.994
TSC + Top-4 (Ours)22.8922.7822.700.74190.59606.4000.05730.02361.127 / 1.406
SPG (TSC + ASC, Ours)22.8922.4122.190.85820.68076.4060.2325 (+54.0%)0.1793 (+69.8%)1.472 / 2.386

Both Table R5a (unseen PickScore evaluation across 300 portfolios) and Table R5b (direct PickScore-guided black-box optimization) confirm that SPG yields consistent, simultaneous improvements in human preference alignment (PickScore), visual aesthetics (Aesthetic V2), and portfolio diversity (CLIP Div, Min Pairwise Dist, Vendi, RKE) without reward hacking.

Reviewer_snRwOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPG, which decomposes test-time image generation into two stages: "candidate search" and "output selection." In the first stage, the search process retains the candidates evaluated; in the second, the image with the highest reward is fixed, and additional images are selected from the pool of candidates that meet a specific reward threshold, ensuring high diversity among the chosen outputs. Experiments conducted across three Stable Diffusion models—with 420 candidates evaluated per prompt—demonstrate that SPG outperforms SPT by improving both the reward of the best-generated image and the CLIP-based diversity of the selected set.

Critical strengths
  1. The problem addressed in the paper is clearly defined, and the methodological framework is both sound and effective. It explicitly separates the process into two stages: "retrieving good candidates" and "selecting an optimal set of results from those candidates." Specifically, the TSC (Targeted Search Component) maintains a search history, while the ASC (Adaptive Selection Component) identifies a set of complementary images from the pool of qualified candidates. This design effectively resolves the issue where selecting only the top-ranked (highest-reward) images might otherwise result in a collection of overly similar outputs.
  2. The experiments are sufficiently comprehensive and include ablation studies on key components. The authors evaluate the approach across three models—SD 1.4, SD 1.5, and SDXL—and the ablation studies successfully isolate the respective contributions of the retrieval and selection stages.
Critical weaknesses
  1. The method lacks innovation and is not sufficiently practical. In essence, SPG builds upon SPT by incorporating components such as Gaussian refreshing and the retention of historical cross-search results. The paper fails to provide a detailed analysis of computational costs; however, given that the approach requires generating 12 images per prompt across 32 iterations of the search process, the computational overhead appears prohibitively high, rendering the method of limited practical value.
  2. The evaluation relies on only a single baseline (SPT), and the same reward model is used for both the search phase and the testing phase, raising concerns about potential "reward hacking." The primary experiments compare the method exclusively against SPT, omitting comparisons with other test-time search techniques (e.g., EvoSearch, FK correctors) or the straightforward "best-of-K" baseline. Furthermore, testing is restricted to older versions of the Stable Diffusion (SD) model family, failing to account for more recent, popular open-source models. The consistent use of ImageReward for both the search and evaluation stages also introduces potential issues regarding the fairness of the assessment.
Questions for authors
  1. Further elaborate on the fundamental innovation of the proposed method.
  2. Include additional baseline comparisons.
  3. Provide a detailed breakdown of computational costs.
  4. Evaluate the method using a broader range of scoring models (e.g., PickScore); does it also yield performance improvements in those cases?
7 comments

We sincerely thank you for the constructive feedback and for recognizing the importance of our research motivation—jointly maximizing reward and output diversity at test time—as well as our cross-model evaluation. We take your critique of the paper's presentation very seriously. Below, we clearly explain (1) the exact objective of test-time portfolio guidance versus single-output optimization, (2) the precise definition and practical necessity of the "black-box reward" setting (even when the reward is parameterized by a deep network), and (3) extensive new experimental results across additional baselines (EvoSearch, SMC / FK-Steering, CADS, Particle Guidance, SPARKE, 10 subset selectors), unseen evaluators (PickScore, LAION Aesthetic V2, CLIP ViT-L/14, Vendi, RKE), candidate budgets (N∈[12,420]N \in [12, 420]), Stable Diffusion 2.1 (768×768), 10-step DPM-Solver++ fast ODEs, and a detailed analysis of applicability to FLUX.

1. Clarifying the Goal of Test-Time Guidance: Single-Output Optimization vs. Portfolio Guidance

Our Clarification & Manuscript Revision: Your understanding is completely correct, and we apologize that the transition in the original Introduction and Section 3 was too abrupt. We have rewritten Section 1 and added a dedicated Section 3.1 (Problem Formulation: Single-Output vs. Portfolio Test-Time Guidance) before introducing our algorithm:

1. Conventional Single-Output Test-Time Optimization (k=1k=1): Given a frozen text-to-image generator Gθ:Z→XG_\theta: \mathcal{Z} \to \mathcal{X} mapping Gaussian source noise z∼N(0,I)z \sim \mathcal{N}(0, I) to an image x=Gθ(z)x = G_\theta(z) for prompt cc, and a scalar reward function R(x,c)∈RR(x, c) \in \mathbb{R}, standard test-time optimization searches for a single image that maximizes reward:

y⋆=arg⁡max⁡x∈PR(x,c)y^\star = \arg\max_{x \in \mathcal{P}} R(x, c)

where P={x1,…,xN}\mathcal{P} = \lbrace x_1, \dots, x_N \rbrace is the set of NN candidate trajectories evaluated during test-time search.

2. Why Single-Output Formulation Fails in Interactive Generation (k>1k > 1): In real-world creative workflows, a text-to-image system returns a small batch / portfolio of kk images (typically k=4k=4) so the user can choose among distinct visual interpretations of an underspecified prompt cc. When iterative search methods (such as SPT, EvoSearch, or SMC / FK-Steering) optimize for high reward, their population collapses into a single local mode in noise space. Returning the top-kk highest-reward candidates from such a search yields kk near-duplicate images (e.g., on SD 1.4, TSC + Top-4 has a pairwise CLIP cosine distance of only 0.0796 on the 20 held-out prompts and 0.1023 on all 100 prompts, while EvoSearch + Top-4 collapses to 0.0622).

3. Our Goal — Test-Time Portfolio Guidance (k>1k > 1): Under a fixed evaluation budget of NN candidate generations, our goal is to return an output subset S={y1,…,yk}⊂PS = \lbrace y_1, \dots, y_k \rbrace \subset \mathcal{P} of size ∣S∣=k|S| = k that simultaneously satisfies three explicit criteria:

  • (i) Peak Reward Preservation (Anchor Optimality): The portfolio must include the single highest-reward candidate discovered during the entire search:

max⁡y∈SR(y,c)=max⁡x∈PR(x,c)=R(x⋆,c)\max_{y \in S} R(y, c) = \max_{x \in \mathcal{P}} R(x, c) = R(x^\star, c)

  • (ii) Worst-Case Quality Floor (Threshold Guarantee): Every returned image in SS must remain within an acceptable reward tolerance τ≥0\tau \ge 0 of the best discovered candidate:

min⁡y∈SR(y,c)≥R(x⋆,c)−τ\min_{y \in S} R(y, c) \ge R(x^\star, c) - \tau

  • (iii) Maximum Inter-Output Diversity (kk-Center Dispersion): Subject to (i) and (ii), the kk returned images should maximize mutual semantic diversity in feature space ϕ(⋅)\phi(\cdot):

max⁡S⊂Pτ, ∣S∣=k, x⋆∈Smin⁡y≠y′∈Sd(ϕ(y),ϕ(y′))\max_{S \subset \mathcal{P}_\tau,\thinspace |S|=k,\thinspace x^\star \in S} \quad \min_{y \neq y^{\prime} \in S} d\big(\phi(y), \phi(y^{\prime})\big)

where Pτ={x∈P:R(x,c)≥R(x⋆,c)−τ}\mathcal{P}_\tau = \lbrace x \in \mathcal{P} : R(x, c) \ge R(x^\star, c) - \tau \rbrace is the quality-certified candidate pool.

4. How SPG Solves This Problem Cleanly in Two Stages:

  • Stage 1 — Trajectory Support Consolidation (TSC): Performs multi-scale archive-driven source-noise search (combining diverse-parent seeding, antithetic mirrored pCN proposals, periodic 8×88 \times 8 block crossover, global Gaussian refresh, and a 3-stage step-size schedule) to drive up the peak reward R(x⋆,c)R(x^\star, c) while consolidating all NN evaluated candidates across all search iterations into the global pool P\mathcal{P}.
  • Stage 2 — Anchored Support Compression (ASC): Fixes y1=x⋆y_1 = x^\star as the anchor and applies farthest-first traversal over Pτ\mathcal{P}_\tau to select the remaining k−1k-1 outputs, achieving a provable 2-approximation to the optimal kk-center dispersion while strictly enforcing (i) and (ii) in 0.181 ms.
  1. Precise Definition and Motivation of "Black-Box Reward"

Your Question: "Does this method only focus on 'black-box rewards'? And how should 'black-box reward' be defined? Does this mean the reward function is not a differentiable function, for example, not parameterized by a deep learning model? Please state this clearly and describe it in detail."

Our Clarification: Thank you for pointing out this ambiguity. In the revised paper (Section 3.1), we explicitly define what "black-box reward" means and explain why our method treats deep-network rewards (like ImageReward, PickScore, or CLIP) as black boxes:

  1. Formal Definition of "Black-Box Reward" (Zeroth-Order Oracle): In test-time diffusion guidance, calling the reward RR a black-box reward means the guidance algorithm accesses the composite objective F(z)=R(Gθ(z),c)F(z) = R(G_\theta(z), c) exclusively through zeroth-order forward evaluations (i.e., passing a candidate image x=Gθ(z)x = G_\theta(z) forward through the evaluator to obtain a scalar score R(x,c)∈RR(x, c) \in \mathbb{R}), without requiring or computing gradients ∇zR(Gθ(z),c)\nabla_z R(G_\theta(z), c) with respect to the noise or latent variables.

  2. Does "Black-Box" Mean RR Cannot Be a Deep Learning Model? No — It Covers Both Non-Differentiable Evaluators AND Deep Neural Reward Models:

    • Case A (Strictly Non-Differentiable Evaluators): The reward may be a non-differentiable rule, discrete OCR/counting verifier, proprietary VLM API, or human preference choice where ∇xR(x,c)\nabla_x R(x, c) does not exist.
    • Case B (Deep Learning Reward Models such as ImageReward, PickScore, or LAION Aesthetic): Even when R(x,c)R(x, c) is parameterized by a differentiable neural network, optimizing the source noise zTz_T via first-order gradients requires computing: ∇zTR(Gθ(zT),c)=(∂Gθ(zT)∂zT)⊤∇xR(x,c)\nabla_{z_T} R(G_\theta(z_T), c) = \left(\frac{\partial G_\theta(z_T)}{\partial z_T}\right)^\top \nabla_x R(x, c) Computing ∂Gθ(zT)∂zT\frac{\partial G_\theta(z_T)}{\partial z_T} requires backpropagating through the VAE decoder and all 50 UNet DDIM denoising steps.
  3. Empirical Proof of Why First-Order Backpropagation Through GθG_\theta Is Prohibitive: We benchmarked the exact GPU memory required to retain the computation graph across 50 DDIM steps on Stable Diffusion 1.4 (batch_size=4, standard pipeline) on an NVIDIA A40 (44.4 GiB usable VRAM) GPU:

Guidance ParadigmGradient Required?Peak GPU VRAM (batch=4, 50 DDIM Steps, SD 1.4)Stability & Compatibility
First-Order Backprop Through Diffusion (∇zTR(Gθ(zT))\nabla_{z_T} R(G_\theta(z_T)))Yes (50 UNet steps + VAE + RR)> 44.42 GiB (CUDA OutOfMemoryError on 48 GB A40)Suffers from vanishing/exploding gradients over 50 steps; requires differentiable RR
SPG Zeroth-Order Black-Box Guidance (@torch.no_grad())No (Forward-only queries)3.12 GiB (> 14.2× lower VRAM)Works identically with neural rewards (ImageReward, PickScore) and non-differentiable evaluators

Thus, treating R(Gθ(z),c)R(G_\theta(z), c) as a black-box forward oracle is not a limitation—it makes SPG >14× more memory-efficient, immune to 50-step gradient instability, and directly compatible with any off-the-shelf neural or non-differentiable evaluator.

3. Expanded Experimental Comparisons, Unseen Evaluators, and Budgets

We conducted an extensive suite of supplementary experiments to thoroughly validate SPG:

A. Multi-Backbone & Cross-Evaluator Generalization (100 Prompts, SD 1.4, SD 1.5, SDXL 1.0)

To verify that SPG does not overfit to the search reward (ImageReward) and generalizes across diverse evaluation metrics, we evaluated the generated portfolios across all 100 prompts on three unseen preference/alignment models (PickScore, LAION Aesthetic V2, OpenAI CLIP ViT-L/14) and two information-theoretic kernel diversity metrics (Vendi Score [TMLR 2023] and RKE Score [NeurIPS 2023]):

A. Multi-Backbone & Cross-Evaluator Generalization (100 Prompts: SD 1.4, SD 1.5, SDXL 1.0)

To verify that SPG does not overfit to ImageReward and generalizes across diverse metrics, we evaluated the portfolios across all 100 prompts on three unseen models (PickScore, LAION Aesthetic V2, OpenAI CLIP ViT-L/14) and two kernel diversity metrics (Vendi Score [TMLR 2023] and RKE Score [NeurIPS 2023]):

BackboneMethod (N=420N=420, k=4k=4)IR (Best / Mean / Min) ↑\uparrowUnseen PickScore (B / M) ↑\uparrowUnseen Aesth. V2 (B / M) ↑\uparrowUnseen CLIP-L/14 (B / M) ↑\uparrowCLIP Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
SD 1.4Standard DDIM (N=4N=4)0.681 / 0.045 / -0.68121.11 / 20.305.839 / 5.49028.09 / 26.250.3701.728 / 2.8691.351 / 2.297
IID (N=420N=420) + Top-41.382 / 1.309 / 1.25221.63 / 21.055.910 / 5.60528.74 / 27.320.3191.622 / 2.6621.289 / 2.094
IID (N=420N=420) + ASC (τ=0.75\tau=0.75)1.382 / 0.963 / 0.71421.67 / 20.895.959 / 5.56828.91 / 27.080.4351.855 / 3.0811.425 / 2.532
SPT (Baseline)1.394 / 1.276 / 1.11021.55 / 21.025.862 / 5.57928.60 / 27.270.3291.645 / 2.7071.303 / 2.144
TSC + Top-4 (τ=0.00\tau=0.00)1.572 / 1.548 / 1.53121.82 / 21.485.864 / 5.65328.25 / 27.390.1021.209 / 1.6211.084 / 1.320
SPG (τ=0.50\tau=0.50, Dom. SPT)1.572 / 1.277 / 1.11221.88 / 21.255.994 / 5.64928.88 / 27.250.3611.709 / 2.8121.339 / 2.241
SPG (τ=0.75\tau=0.75, Default)1.572 / 1.140 / 0.89421.89 / 21.146.020 / 5.64328.99 / 27.140.4091.815 / 3.0121.401 / 2.452
SPG (τ=1.00\tau=1.00, Dom. IID+ASC)1.572 / 0.992 / 0.65121.91 / 21.026.038 / 5.62929.04 / 27.110.4381.864 / 3.0981.431 / 2.551
SD 1.5SPT (Baseline)1.385 / 1.257 / 1.08521.74 / 21.235.961 / 5.67728.56 / 27.270.3231.630 / 2.6681.295 / 2.114
SPG (τ=0.75\tau=0.75, Default)1.542 / 1.111 / 0.85921.98 / 21.266.077 / 5.71828.88 / 27.190.4041.813 / 3.0111.400 / 2.448
SDXL 1.0SPT (Baseline)1.441 / 1.327 / 1.17422.46 / 21.966.440 / 6.16629.21 / 28.120.2571.510 / 2.4111.227 / 1.855
SPG (τ=0.75\tau=0.75, Default)1.513 / 1.079 / 0.79122.66 / 21.996.544 / 6.18529.59 / 27.990.4551.815 / 3.0211.402 / 2.456
  • Note that SPG (τ=0.50\tau=0.50) strictly dominates SPT on every metric simultaneously across all 100 prompts (Best, Mean, Min ImageReward, unseen PickScore, Aesthetic V2, CLIP-L/14, CLIP Div, Vendi, RKE), while SPG (τ=1.00\tau=1.00) strictly dominates IID + ASC (τ=0.75\tau=0.75) on every metric simultaneously!

B. Additional Search (EvoSearch, SMC / FK-Steering, SPT + History + ASC) & Diversity Baselines (CADS, Particle Guidance, SPARKE) (Held-Out 20 Prompts, SD 1.4)

MethodNNIR Best ↑\uparrowIR Mean ↑\uparrowMin ↑\uparrowCLIP Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
Standard DDIM (In-Sampling)40.69290.0445-0.68120.35191.690 / 2.8381.326 / 2.248
CADS (ICLR'24, Reward-Blind)40.69870.0362-0.65400.38871.751 / 2.9761.367 / 2.418
Particle Guidance (ICLR'24)40.72790.0970-0.58120.36271.710 / 2.8851.339 / 2.299
SPARKE (NeurIPS'25)40.67800.0937-0.59040.36521.715 / 2.8931.342 / 2.310
SMC / FK-Steering (Terminal)4201.28211.21371.12400.15081.311 / 1.9841.121 / 1.501
EvoSearch (Top-4 Elites)†^\dagger4201.70291.69231.68150.06221.138 / 1.4121.052 / 1.198
SPT (Terminal Cold Chain)4201.46791.35631.21840.25831.520 / 2.3881.236 / 1.907
SPT + History + ASC (τ=0.75\tau=0.75)4201.54051.10070.85940.36841.727 / 2.8611.352 / 2.311
TSC + Top-4 (Ours)4201.61701.60221.59210.07961.163 / 1.4911.064 / 1.246
SPG (τ=0.50\tau=0.50, Strict Dom.)4201.62361.32491.16400.28401.565 / 2.5481.252 / 2.004
SPG (τ=0.75\tau=0.75, Default)4201.61701.18710.93960.35991.702 / 2.8001.336 / 2.227
SPG (τ=1.00\tau=1.00, High Div.)4201.62361.04980.70310.37021.730 / 2.8621.354 / 2.311

(Note: †^\daggerTerminal EvoSearch collapses portfolio diversity to Dψ=0.0622D_\psi = 0.0622, returning four near-identical clones.)

Reviewer_uHXyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies test-time reward guidance for text-to-image diffusion models with black-box rewards, where many candidates are generated by searching over the source noise but only a few images are returned. The authors propose Source Portfolio Guidance (SPG), which has two parts: Trajectory Support Consolidation (TSC), which keeps a history of all evaluated candidates alongside a reward-ranked archive used for pCN-based search, and Anchored Support Compression (ASC), which keeps the best candidate and fills the remaining slots by farthest-first selection among candidates within a reward threshold.

Critical strengths
  • The observation that top-k selection from a reward-driven search returns redundant images is practical and relevant.

  • ASC is simple and does not require additional generator evaluations.

  • The same-pool ablation (Table 3) nicely isolates the effect of the selection step from the search step.

  • Results are reported on held-out prompts with paired confidence intervals.

Critical weaknesses

My main concern regarding this work is its originality. TSC essentially stores all evaluated candidates instead of only the current archive, and ASC is a reward-thresholded farthest-first traversal (Gonzalez, 1985) anchored at the best sample. Both are standard techniques, and the guarantees (keeping the best reward, the reward floor, and the 2-approximation of K-center) follow directly from the construction, as the authors also acknowledge. Diversity-aware subset selection has been widely studied (e.g., MMR [7] and DPPs), and the paper does not clearly explain what is new beyond applying these tools to the outputs of reward search. Below I outline additional concerns:

  • Missing baselines for selection. Since the main gain in diversity comes from ASC, it should be compared on the same candidate pool with other subset selection methods, such as greedy k-DPP (already cited in the paper), MMR [7], or greedy selection based on diversity scores such as Vendi [5] or RKE [6]. Also, applying ASC to the history of SPT is missing, which is needed to understand whether TSC's search itself is better than SPT's.

  • Missing baselines for diverse generation. Diversity guidance methods such as Particle Guidance [1], CADS [2], c-VSG [3], and SPARKE [4] generate diverse samples directly during sampling, at a much lower cost than searching over 420 candidates. SPARKE in particular also uses the history of previously generated samples, which is close in spirit to TSC, but uses it to guide generation rather than only for post-hoc selection. EvoSearch is cited but also not compared.

  • The SPT baseline seems weak. In Table 3, IID sampling with ASC reaches a best reward of 1.382 (vs. 1.394 for SPT) with much higher diversity (0.435 vs. 0.329). This suggests that a simple best-of-N with diversity-aware selection is already competitive, and comparing only with SPT may overstate the improvement.

  • Computational cost. Each prompt requires 420 full generations with 50 DDIM steps (21,000 sampling steps) plus reward evaluation to return only four images. Only a single budget is evaluated, and no wall-clock time is reported in the paper. A study of how performance changes with smaller budgets (e.g., 50, 100, 200 candidates), and a runtime comparison with SPT and diversity-guidance methods, would clarify the practical value of the method.


[1] Corso et al. "Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models", ICLR 2024

[2] Sadat et al. "CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling", ICLR 2024

[3] Askari Hemmat et al. "Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance", ECCV 2024

[4] Jalali et al. "SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score", NeurIPS 2025

[5] Friedman and Dieng. "The Vendi Score: A Diversity Evaluation Metric for Machine Learning", TMLR 2023

[6] Jalali et al. "An Information-Theoretic Evaluation of Generative Models in Learning Multi-modal Distributions", NeurIPS 2023

[7] Carbonell and Goldstein. "The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries", SIGIR 1998

Questions for authors
  • How does SPG compare with applying ASC to the candidate history of SPT?

  • How do the results change with smaller candidate budgets, and what is the wall-clock time per prompt compared with SPT?

  • How does farthest-first selection compare with DPP- or Vendi/RKE-based subset selection on the same candidate pool?

  • What is the set-mean ImageReward of SPT, and how does SPG perform under preference models that are not used during search?

8 comments

We sincerely thank you for the thorough, constructive review and for recognizing that (1) identifying top-kk redundancy in reward-guided search is practical and well-motivated, (2) ASC requires zero extra generator calls, (3) our same-pool ablation cleanly isolates search from selection, and (4) our held-out evaluation with paired confidence intervals is rigorous.

To address every question directly, we conducted extensive new experiments on NVIDIA A40 GPUs, adding:

  1. Same-pool comparison of 10 subset selection algorithms (Top-4, Unthresholded Farthest-First, MMR (λ=0.5, 0.7), Anchored MMR, Greedy k-DPP (QD), Anchored k-DPP, Anchored Vendi, Anchored RKE, and ASC (Ours));
  2. Applying ASC (and k-DPP/Vendi/RKE) directly to SPT's full historical candidate pool (SPT + History + ASC) across 9 candidate budgets (N∈[12,420]N \in [12, 420]);
  3. Training-free diversity-guidance baselines (CADS [ICLR 2024], Particle Guidance [ICLR 2024], SPARKE [NeurIPS 2025]) and equal-budget search baselines (EvoSearch and SMC / FK-Steering at N=420N=420);
  4. Full τ\tau-Pareto analysis across all 100 prompts (τ∈[0,1.50]\tau \in [0, 1.50]), demonstrating strict simultaneous dominance over both SPT (at τ=0.50\tau=0.50) and IID + ASC (at τ=1.00\tau=1.00);
  5. Evaluation on 5 unseen metrics (PickScore, LAION Aesthetic V2, CLIP ViT-L/14, Vendi Score [TMLR 2023], and RKE Score [NeurIPS 2023]) across all 100 prompts on SD 1.4, SD 1.5, and SDXL 1.0, plus direct PickScore-guided black-box search and Stable Diffusion 2.1 (768×768) generalization;
  6. Multi-budget scaling curves (N∈{12,48,96,144,192,240,300,360,420}N \in \{12, 48, 96, 144, 192, 240, 300, 360, 420\}), fine-grained timing breakdown, and 10-step DPM-Solver++ fast ODE generalization.

1. Originality & Conceptual Contribution (Weakness 1)

We clarify our core conceptual and algorithmic contributions beyond standard components:

  1. Reframing Test-Time Reward Guidance from Terminal-State Sampling to Trajectory-Support Portfolio Optimization:

    • Existing test-time search methods (SPT, SMC / FK-Steering, EvoSearch) treat the search trajectory as a discardable burn-in path, returning only the terminal population XTX_T. Under high inverse temperature βC\beta_C or elitist resampling, XTX_T inevitably collapses into a single local reward basin.
    • We show that the union of all evaluated candidates along a search trajectory PT=⋃t=0TXt\mathcal{P}_T = \bigcup_{t=0}^T X_t forms a rich, multimodal reward-elevated support. However, simply saving SPT's history (SPT + History) is insufficient because SPT's local Metropolis-Hastings chains get trapped early.
    • TSC is not merely "saving history": TSC replaces SPT's coupled MH chains with a non-stationary, archive-driven multi-scale proposal mechanism combining (i) diverse-parent archive seeding (selecting the highest-reward anchor z(a)z^{(a)} and furthest elite latent z(b)z^{(b)} in the top-16 archive), (ii) antithetic mirrored pCN proposals (z′=1−s2z±sξz' = \sqrt{1-s^2} z \pm s \xi), (iii) periodic spatial block crossover (8×88 \times 8 latent patch exchange every 5 rounds), (iv) persistent global Gaussian refresh, and (v) a 3-stage step-size schedule (explore →\to consolidate →\to refine). In Table R2, under the same selector, TSC outperforms SPT + History by +0.0765 Best Reward and +0.0864 Mean Portfolio Reward (and terminal SPT by +0.1491 Best Reward).
  2. Why Quality-Thresholded kk-Center (ASC) Outperforms Soft Additive (MMR) and Determinant/Entropy (k-DPP, Vendi, RKE) Selectors for Portfolio Generation:

    • Standard subset selectors like MMR (λR(x)+(1−λ)div(x)\lambda R(x) + (1-\lambda)\text{div}(x)) or Greedy k-DPP (Lij=qiKijqjL_{ij} = q_i K_{ij} q_j) trade off quality and diversity via a soft additive or multiplicative weight. When candidate rewards span [−2.0,+2.0][-2.0, +2.0], soft trade-offs fail: they willingly admit a severely deformed, low-reward outlier if its feature distance from the anchor is large enough, or conversely collapse diversity when exponential quality weights qi=exp⁡(αRi)q_i = \exp(\alpha R_i) dominate the determinant.
    • By contrast, ASC enforces a hard worst-case quality certificate (min⁡x∈SR(x)≥R(x∗)−τ\min_{x \in S} R(x) \ge R(x^*) - \tau, retaining x∗x^* exactly) and solves the metric kk-center dispersion problem inside the certified superlevel set Eτ\mathcal{E}_\tau. Unlike sum-diversity (MMR) or volume/entropy (k-DPP/Vendi/RKE), which can place two samples close together if the average determinant is large, kk-center farthest-first traversal maximizes the minimum pairwise distance min⁡x≠y∈Sd(x,y)\min_{x \neq y \in S} d(x, y), guaranteeing that no two returned images in the K=4K=4 portfolio are near-duplicates.

Q3: We evaluated 10 subset selection algorithms on the exact same 420-candidate pools generated by TSC across all 20 held-out prompts (SD 1.4, K=4K=4), measuring ImageReward (Best, Mean, and Worst-Case Min min⁡x∈SR(x)\min_{x \in S} R(x)), CLIP ViT-L/14 pairwise diversity, BLIP pairwise diversity, Vendi Score [5] (Cosine & RBF kernels), RKE Score [6] (Cosine & RBF kernels), and selection wall-clock runtime:

Table R1: Same-Pool Comparison of 10 Subset Selection Algorithms (N=420N=420, Held-Out 20 Prompts)

Subset Selection Method (Same TSC Pool, N=420N=420)Best Reward ↑\uparrowMean Reward ↑\uparrowWorst-Case Min Reward ↑\uparrowCLIP-L/14 Div ↑\uparrowBLIP Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrowSelection Time (ms) ↓\downarrow
Top-4 (Reward-Only)1.61701.60221.59210.07960.05051.163 / 1.4911.064 / 1.2460.057 ms
Unthresholded Farthest-First (Gonzalez)†^\dagger1.6170-0.2214-1.48820.49330.72951.954 / 3.1871.494 / 2.6710.102 ms
MMR (λ=0.5\lambda = 0.5) [7]†^\dagger1.61701.15840.71110.41280.50561.804 / 2.9431.402 / 2.3980.080 ms
MMR (λ=0.7\lambda = 0.7) [7]1.61701.45191.29150.32560.34781.634 / 2.6641.298 / 2.1000.062 ms
Anchored MMR on Eτ\mathcal{E}_\tau (τ=0.75\tau=0.75)1.61701.45731.30630.31090.33751.607 / 2.6281.280 / 2.0610.286 ms
Greedy kk-DPP (Quality-Diversity LL-Ensemble)1.61701.51681.42790.25520.27451.504 / 2.4181.221 / 1.87019.302 ms
Anchored Greedy kk-DPP on Eτ\mathcal{E}_\tau (τ=0.75\tau=0.75)1.61701.49321.37670.27490.30541.543 / 2.5041.242 / 1.9517.790 ms
Anchored Greedy Vendi Score on Eτ\mathcal{E}_\tau (τ=0.75\tau=0.75) [5]1.61701.19380.91790.36620.43631.710 / 2.8211.339 / 2.25113.384 ms
Anchored Greedy RKE Score on Eτ\mathcal{E}_\tau (τ=0.75\tau=0.75) [6]1.61701.18790.91760.36790.43551.716 / 2.8221.344 / 2.2545.403 ms
ASC (τ=0.50\tau = 0.50, Strict-Dominance Setting)1.62361.32491.16400.28400.34121.565 / 2.5481.252 / 2.0040.174 ms
ASC (τ=0.75\tau = 0.75, Default Ours)1.61701.18710.93960.35990.43061.702 / 2.8001.336 / 2.2270.181 ms
ASC (τ=1.00\tau = 1.00, High-Diversity Ours)1.62361.04980.70310.37020.46411.730 / 2.8621.354 / 2.3110.184 ms

(Note: †^\daggerViolates the worst-case quality floor Rmin⁡≥R(x∗)−τR_{\min} \ge R(x^*) - \tau, admitting degraded outliers.)

Key conclusions from Table R1:

  1. Unthresholded Farthest-First and Soft MMR (λ=0.5\lambda=0.5) suffer from severe worst-case quality drops: Without our hard threshold Eτ\mathcal{E}_\tau, Unthresholded-FF collapses Mean Reward to -0.2214 (Worst-Case Min = -1.4882), and MMR ($\lambda=0.5$) drops Worst-Case Min Reward to 0.7111 (-0.2285 lower than ASC), frequently admitting degraded outliers.
  2. Standard Greedy k-DPP and Anchored k-DPP under-diversify: Because kk-DPP optimizes log-determinant volume weighted by quality, it clusters closer to the reward peak, achieving only 0.2552–0.2749 CLIP diversity (24%–29% lower than ASC's 0.3599) while being 43×–106× slower.
  3. Once restricted to our anchored superlevel set Eτ\mathcal{E}_\tau (τ=0.75\tau=0.75), ASC matches Anchored Greedy Vendi and Anchored Greedy RKE within <1.8% on all diversity metrics while achieving a HIGHER worst-case minimum reward (0.9396 vs. 0.9179 / 0.9176), the HIGHEST kk-center minimum pairwise distance when selecting in CLIP space (0.3495 vs. 0.3424 for Anchored k-DPP, 0.3395 for Anchored RKE, and 0.3288 for Anchored Vendi), and running 30×–74× faster (0.181 ms vs. 5.40–13.38 ms) with zero kernel bandwidth hyperparameters.

Q1: We recorded all 420 candidates evaluated across all 3 chains and 35 rounds of SPT (SPT + History) and applied Top-4 and ASC ($\tau=0.75$) directly to SPT's historical pool on the same 20 held-out prompts:

Table R2: Disentangling Search (SPT vs. TSC) and Selection (Terminal vs. Top-4 vs. ASC) at N=420N=420

Search Trajectory + Selection Strategy (N=420N=420)Best Reward ↑\uparrowMean Reward ↑\uparrowWorst-Case Min Reward ↑\uparrowCLIP-L/14 Div ↑\uparrowVendi (Cos / RBF) ↑\uparrowRKE (Cos / RBF) ↑\uparrow
SPT (Standard Terminal Cold Chain)1.46791.35631.21840.25831.520 / 2.3881.236 / 1.907
SPT + History + Top-41.54051.51941.50120.15571.306 / 1.7981.127 / 1.505
SPT + History + ASC (τ=0.75\tau=0.75)1.54051.10070.85940.36841.727 / 2.8611.352 / 2.311
TSC + Top-4 (Our Search + Top-4)1.61701.60221.59210.07961.163 / 1.4911.064 / 1.246
SPG (τ=0.50\tau = 0.50, Strict-Dominance Ours)1.62361.32491.16400.28401.565 / 2.5481.252 / 2.004
SPG (τ=0.75\tau = 0.75, Default Ours: TSC + ASC)1.61701.18710.93960.35991.702 / 2.8001.336 / 2.227
SPG (τ=1.00\tau = 1.00, High-Diversity Ours)1.62361.04980.70310.37021.730 / 2.8621.354 / 2.311

Key Takeaways from Table R2:

  1. Applying our ASC selector to SPT's history (SPT + History + ASC) already boosts SPT's Best Reward from 1.4679 to 1.5405 (+0.0726) and CLIP Diversity from 0.2583 to 0.3684 (+42.6%), confirming that historical consolidation + ASC is a general upgrade even for standard MCMC chains.
  2. However, TSC search is substantially stronger than SPT + History: Under the exact same selection rule, TSC outperforms SPT + History by +0.0765 Best Reward (1.6170 vs. 1.5405), +0.0828 Top-4 Mean Reward (1.6022 vs. 1.5194), +0.0864 ASC Mean Portfolio Reward (1.1871 vs. 1.1007), and +0.0802 Worst-Case Min Reward (0.9396 vs. 0.8594). Furthermore, as shown in Table R5 below, TSC surpasses SPT + History at every single candidate budget from N=48N=48 to N=420N=420.

Comparison with Training-Free Diversity Guidance (CADS, Particle Guidance, SPARKE) and Equal-Budget Search (EvoSearch, SMC / FK-Steering) (Weakness 3)

We implemented and evaluated the requested training-free in-sampling diversity guidance methods — CADS [2] (ICLR 2024), Particle Guidance [1] (ICLR 2024), and SPARKE [4] (NeurIPS 2025, which uses historical RKE kernel gradients to guide generation) — as well as equal-budget black-box search baselines EvoSearch and SMC / FK-Steering (N=420N=420 candidates) on all 20 held-out prompts:

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.