acceptodds
Under review as a conference paper at ICLR 2027

Sampling Meets Interaction: Sequentially-Controlled Multi-Particle Flow-Maps for Efficient Inference-time Search

Abstract

While generative models have enabled training-free reward alignment, existing particle-based methods are fundamentally constrained by their propensity to local exploration within narrow regions of the underlying distribution, severely restricting sample diversity. This limitation becomes especially acute under tight reward-feedback budgets, where effective search demands broad, strategic exploration to uncover high-utility regions. To address this, we propose Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for feedback-efficient search. IMPFM progressively transports a group of interactive particles toward the target distribution, maintaining the broad coverage essential for heterogeneous preference alignment. IMPFM leverages a principled and efficient posterior sample-sharing mechanism across particles powered by flow maps. By correcting individual particle drift with the collective value gradient from the entire ensemble's posterior samples at each correction step, the framework maximizes sample utility to enable global exploration while actively mitigating reward over-optimization, typical of standard control frameworks. Paired with a principled exploration-exploitation reweighting mechanism involving multi-particle interaction, this sequentially corrected multi-particle dynamics explicitly preserves structural diversity and overcomes the weight degeneracy inherent to standard Sequential Monte Carlo (SMC) samplers. Crucially, we prove that the resulting sampling framework yields a multi-particle interaction-aware Feynman-Kac corrector that progressively steers the multi-particle system toward a KL-tilted target distribution, facilitating global exploration and preventing mode collapse. Extensive empirical evaluations and ablations across diverse search and alignment tasks confirm the efficacy of IMPFM over existing baselines.

Primary area:
generative models
Keywords:
Inference Time Search, Flow Maps, Feynman-Kac Corrector

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_WgUXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IMPFM, a multi-particle sampling framework for feedback-efficient inference-time search. The method combines flow-map-based posterior sampling with value-guided particle updates and kernel-based interactions. It also introduces an interaction-aware reweighting mechanism to improve sample diversity and reduce weight degeneracy.

The authors evaluate IMPFM on ImageNet-based search and text-to-image alignment tasks, including compositional and counting prompts. The results show improvements in reward scores and sample diversity over several SMC-based and reward-guided sampling methods under the same feedback budget.

Critical strengths
  1. The author proposes an interesting approach to multi-particle guidance. The paper combines value-based guidance with particle interactions to balance reward optimization and diversity. Sharing information across particles is a useful idea, especially when reward feedback is expensive.

  2. The paper gives promising experimental results. IMPFM shows clear improvements over several baselines across both search and alignment tasks. The ablation studies also provide useful evidence for the benefits of particle interaction and reweighting.

Critical weaknesses
  1. Unclear KL-tilted target. Proposition 4.3 defines the target using a KL divergence between distributions, which is a scalar rather than a function of xx. It is unclear how this produces the claimed reweighting.

  2. Questionable SVGD derivation. Appendix A modifies the normalization of the standard SVGD update, but without sufficient justification. It is unclear whether the resulting update still gives the claimed steepest descent direction.

  3. Lack of posterior sampling guarantees. The iterative sampling procedure in Eq. (12) is not theoretically justified. Do the generated samples follow the desired conditional posterior?

  4. Unclear importance weighting. The scores in Eq. (9) are not necessarily positive. The paper does not clearly explain how they are converted into valid importance weights for resampling.

  5. Limited computational and statistical evaluation. The experiments mainly compare feedback budgets, without reporting comparable runtime or computational costs.

Questions for authors
  1. Could the authors clarify the definition of the KL-tilted distribution in Proposition 4.3? In particular, how does the proposed KL term produce an xx-dependent reweighting?

  2. Why do the particle updates in Eqs. (6) and (9) differ from those in Eqs. (10) and (11)? Could the authors also explain the normalization used in Appendix A?

  3. What guarantees that the iterative procedure in Eq. (12) samples from the conditional posterior associated with the original xtx_t? Is there any empirical validation of this approximation?

  4. How are the scores in Eq. (9) converted into nonnegative importance weights? Are exponentiated log-weights used to compute ESS and perform resampling?

  5. Could the authors provide comparisons under matched computational budgets? This would help clarify the practical efficiency and stability of IMPFM.

Reviewer_RkWVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SVGD style evolution of interacting particles with repulsion for reward steering. Flow maps are exploited for more efficient evaluations. Various additional adjustments (e.g., stochasticity, weighting) are introduced presumably to make it work in practice. The presentation could be improved substantially.

Critical strengths

SVGD style evolution of interacting particles with repulsion for reward steering seems worth exploring further though the approach as stated would inherently work only in low dimensions.

Critical weaknesses

Writing lacks clarity, structure and polish. Lots of little errors and unsubstantiated claims. The basic idea quickly dissolves into a collection of patches that appear increasingly heuristic.

It would help to strip away all the unnecessary parts and evaluate each key step carefully. For example, additional noise injection seems necessary for resampling to refocus particles on more promising paths and to differentiate their paths in a controlled manner from there onwards. Repulsion alone doesn't guarantee exploration in high dimensions. Why is it included at all?

Proposition 4.3 seems wrong. KL is a scalar so the proposed tilting isn't tilting at all.

there's no mention of the difficulty of using radial basis kernels in high dimensions such as images. How was the bandwidth chosen?

The sampling scheme in Eq (12) is not sampling from the posterior since the connection to x_t is lost after the first iteration. What is it meant to do??

Questions for authors

see above.

Not clear why one wouldn't use p_t exp(V_t) as the target in SVGD and link the particle evolution to flow schedule as opposed to using exp(V_t) as the target alone; the score for p_t can be obtained from the flow model.

Reviewer_LxkUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes IMPFM, an inference-time search method under limited budgets. It combines conditional flow-map sampling with cross-particle info sharing to improve reward while preserving diversity. Experiments evaluate the method on ImageNet search, compositional alignment, and quantity-aware image generation.

Critical strengths

The particle interaction design is well motivated. Sharing value-gradient information across particles plush introducing repulsive interactions is a clever design. This helps to address the challenge in inference-time search under limited feedback budgets.

Empericial results are strong. The reported improvements in reward and diversity across three image-generation settings support the method’s practical value. Evaluation using held-out reward models also provides evidence that the improvements extend beyond optimizing the guidance reward alone.

Critical weaknesses

Some bugs in theory part. In Proposition 4.3, the proposed target is ptKL(x)∝qt(x)exp⁡[−DKL(qt∥pboltz)]p_t^{KL}(x)\propto q_t(x)\exp[-D_{KL}(q_t\|p_{\mathrm{boltz}})]. Under the standard definition, the KL divergence is a scalar independent of xx, so this factor cancels upon normalization, leaving ptKL=qtp_t^{KL}=q_t. Appendix B nevertheless treats this distribution-level functional as a pointwise potential. Then it substitutes an SVGD transport direction for its spatial gradient without establishing the equivalence. Separately, Appendix A, Step 5 changes the relative coefficients of the self-gradient and other-particle terms. This is not an overall rescaling of the preceding RKHS steepest-descent direction. Consequently, the derivation does not establish the stated connection between the actual update and that optimization principle. They do not negate the empirical improvements, but they weaken the paper’s claims.

Missing some important ablation study. The relevant comparisons, including adding or removing gradient sharing and repulsive interactions together. They therefore support the interaction mechanism as a whole, but do not establish whether feedback from one trajectory benefits related trajectories beyond the effect of repulsion alone. This distinction is central to the paper’s explanation of improved feedback efficiency. Sharing-only, repulsion-only, and combined variants, with other components fixed and matched particle counts and total reward-evaluation budgets.

minors. The Figure2 can be improved, the lines and dotted lines are overlap. The figure can be better arranged. Too many symbols and formulas in main text, I suggest to move to appendix (even you already did) , just leave some summarized or key parts. Missing the title for Figure5 and the line plots are too small.

Questions for authors

See weakness part. I'm willing to raise my score if the author adress my concerns. I prefer to give 2.5 but we don't have such option...

Reviewer_3QmgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IMPFM for generative-model search and alignment under limited feedback budgets. The method uses conditional posterior flow maps to estimate soft value-function gradients and augments individual particle control with kernel-weighted gradients from other particles and a repulsive interaction term. These updates are combined with stochastic transitions, reweighting, and resampling to improve reward alignment and sample diversity. The authors evaluate IMPFM on ImageNet search, compositional alignment, and quantity-aware alignment, reporting improvements over several existing baselines.

Critical strengths

The paper provides a reasonably broad empirical evaluation across tasks with different challenges. Specifically, the experiments include 200 randomly selected ImageNet classes for search, 200 compositional prompts from GenAI-Bench, and 200 numeracy prompts from T2I-CompBench++. These settings cover fine-grained attributes, complex semantic composition, and object counting, providing useful evidence about the method’s behavior across different alignment requirements. The evaluation considers several complementary aspects of performance, including reward and alignment scores, LPIPS for perceptual diversity, ESS for particle-weight concentration, and qualitative visualizations. Together, these measurements help characterize the relationship between reward improvement, sample diversity, and particle dynamics.

Critical weaknesses
  1. The ablations do not sufficiently isolate the contribution of posterior information sharing. The method combines kernel-weighted attraction, repulsion, reweighting, and resampling. Fig. 5a removes interaction from both drift correction and reweighting, while Fig. 6 removes reweighting altogether. These comparisons do not isolate the individual contributions or directly demonstrate that information sharing improves value-gradient estimation under a fixed feedback budget.

  2. Eq. (12) does not establish sampling from the posterior conditioned on the original state. Each renoising step changes the conditioning state, so the outputs cannot generally be treated as samples from the posterior conditioned on the original intermediate state. The authors should justify this posterior interpretation or clarify the approximation involved, since these samples determine the value-gradient estimates and subsequent updates.

  3. The feedback-budget accounting is insufficient to establish fair comparisons. Appendix J specifies 32 particles and one posterior sample per particle for search, but does not report corresponding configurations for each baseline. Without particle counts, posterior sample counts, total reward evaluations, and output-selection rules, the reported “Feedback Budgets” do not establish equivalent resources, particularly for the comparison with MFM.

Questions for authors
  1. Can the authors independently evaluate interaction in drift correction and interaction in reweighting under the same total feedback budget? What evidence specifically supports improved value-gradient estimation through information sharing?

  2. What guarantees that the outputs of Eq. (12) follow the posterior conditioned on the original intermediate state, despite the conditioning state changing during iteration? Is this procedure intended to be exact or approximate?

  3. Can the authors provide the particle count, posterior samples per particle, feedback steps, total reward evaluations, and output-selection rule for each baseline, and explain precisely how these quantities map to the reported feedback budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.