acceptodds
Under review as a conference paper at ICLR 2027

Coverage-Constrained Neural Exponential Embedding: Exact Test-Time Projection for Frozen Generators

Abstract

Test-time control of a generator is often implemented by prompting, scalar guidance, or reweighting a candidate bank. These mechanisms can improve a preferred attribute while silently concentrating probability on very few outputs. We instead ask for the closest distribution to a frozen generator that matches target statistics to a declared tolerance and retains a minimum effective sample size. The resulting finite-bank problem is a convex, coverage-constrained information projection. We prove existence and uniqueness, derive its low-dimensional dual, and show that an active coverage constraint changes the density ratio from an exponential tilt to a Lambert-W deformation. Thus classical empirical tilting is recovered only when coverage is inactive; rescaling a fitted tilt along a ray is generally not the constrained optimum. A reference-only bootstrap calibrates moment tolerance without evaluation leakage. The fitted density ratio can be used directly for bank resampling or as the terminal potential of learned twisted sequential Monte Carlo. Across frozen image and language generators, the exact projection improves held-out distribution matching over ray temper- ing, remains competitive with density ratio and stable-balancing baselines, and produces substantially better diffusion targets at matched fit-bank coverage. The method separates target specification, optimal projection, and sequential realization, making both its guarantee and its support limitations explicit.

Primary area:
generative models
Keywords:
Neural Exponential Embedding, Test-Time Learning, Frozen Generator, Moment Matching

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8ghNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper adapts frozen generative models by reweighting samples to approximately match target feature averages while maximizing the entropy of the weights. They impose a minimum effective sample size and derive a Lambert-W solution. They evaluate reweighting on image and text generators and show learned twisted sequential Monte Carlo sampling on a CIFAR-10 diffusion model.

Critical strengths

The paper combines approximate entropy balancing with an explicit ESS constraint, equivalent to a lower bound on Renyi-2 entropy, which the second part is an interesting addition.

They also connect the fitted density ratio to learned twisted Monte Carlo sampling for sequential generation.

Critical weaknesses
  • The core theoretical ingredients are well established, including entropy balancing and the Lambert-W solution for entropy-plus-quadratic objectives under moment constraints. The explicit ESS constraint is interesting, but the novelty beyond combining these ingredients is unclear to me.

  • I am not yet convinced of the method's practical usefulness. The method matches averages of selected per-sample features, and it is unclear whether this linear combination can capture the aspects of the target distribution that matter in practice. Weight entropy and ESS do not account for similarity between samples, so distributing weight across many nearly identical samples can yield high entropy and ESS without meaningful semantic diversity.

  • The experimental scope is limited. In particular, sequential sampling is demonstrated only on a CIFAR-10 brightness task, so its usefulness for more complex image and text generation remains unclear. Moreover, the improvements in Table 2 over the stronger baselines seem marginal.

Questions for authors
  • Could you justify the choice of ESS as the coverage criterion and explain how its minimum should be chosen in practice?

  • What practical applications would benefit from this method? Like could the method target generative precision/density, and what effect would this have on recall/diversity?

Reviewer_zaxtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a constrained importance-reweighting method for frozen generative models (e.g., diffusion models and LLMs), which optimizes weights over generated samples to approximately match the statistics of a desired target distribution while maintaining a minimum effective sample size (ESS). This is formulated as a KL-minimizing convex optimization problem, for which the authors derive a Lambert-W characterization of the optimal weights and a low-dimensional dual solution. The resulting weights can be used for direct sample resampling or as terminal potentials for SMC-based generation. Experiments on image and language generators, including SDXL, Qwen, and CIFAR diffusion models, report improvements over conventional exponential-tilt tempering on selected distribution-matching tasks.

Critical strengths

The theoretical formulation especially related to optimization looks quite sound to me.

Critical weaknesses
  1. Unclear motivation and practical significance: My biggest concern is the lack of a clearly motivated application. While the mathematical formulation is well-defined, it is unclear why reweighting the outputs of a frozen generative model to match a target distribution is particularly useful. Surprisingly, the paper does not provide a single concrete qualitative example, despite working with image-generation models and LLMs. Even in the SDXL experiments, the exact source/target prompts, before-and-after generations, and qualitative comparisons against baselines are absent. This makes it extremely difficult to understand what the method actually achieves. While I can see the motivation for controlling an unconditional diffusion model (e.g., CIFAR), I struggle to understand the practical value of reweighting a frozen LLM's outputs to achieve prescribed gender proportions, rather than directly controlling the prompt or sampling the desired categories.

  2. Insufficient engagement with existing literature: There is extensive literature on test-time generative control, including best-of-N, noise optimization, reward-guided sampling, SMC, and distributional guidance, with concrete applications across image, video, language, and scientific generation. Although the paper cites some related methods, it neither meaningfully compares against them nor establishes a compelling application where its approach would be advantageous. Instead, the experiments primarily compare alternative statistical reweighting techniques, without demonstrating an advantage over established generative-control methods.

  3. Limited theoretical novelty: The proposed KL-minimizing importance reweighting with an ESS constraint is closely related to classical entropy balancing and constrained importance weighting. The Lambert-W characterization follows directly from the KKT conditions. While the exact formulation may be new, its substantive theoretical novelty is unclear.

Questions for authors

As mentioned earlier, while the theoretical formulation appears sound (even if it's a direct application of KKT conditions to this task), it would be great if the authors could clarify the intended goals of the paper? Could the authors think of a better and more relevant task to illustrate the benefits of their proposed optimization method? Could there be better comparisons to validate the benefits of the proposed approach?

Reviewer_fkbqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a test-time distribution control based on a frozen generative models. Specifically, a candidate bank is first constructed by the original sampling procedure from the fixed generator. Then, probabilities are assigned to the candidate images so the weighted sample features match certain target statistics. A new sampling distribution can then be obtained via sampling from the bank according to the assigned probability weights.

Critical strengths
  1. The formulationof the tradeoff between distribution matching and resultant sample diversity to a convex optimization problem is clear and sound. Moreover, the theoretical discussoin on the exponential tilting is interesting. Instead of fitting a conventional exponential tilt and then weaken it (via say ray tempering) to ensure the diversity on resultant sampling, the proposed method jointly optimizes both objectives, which is a systematic improvement over the original exponential tilting.
  2. The experiments support the theoreticaly claimed advantage, covering multiple data modalities. The comparisons are well controlled to isolate the comparsion between the proposed Lambert-W projection against the conventional ray tempering.
Critical weaknesses
  1. There is an important gap between what the theory justifies and what the method is claimed to achieve: the reweighted distribution is theoretically optimal regarding to the fixed set of samples (the bank). However, it doesn't guarantee that the method would still lead to the controlled distribution from the generator to be optimal for newly generated samples.
  2. Similarly, the learned desnity-ratio function R(x) is simply evaluated using the fitted parameters on the new samples, and thus also not guaranteed to generalize to new samples.
Questions for authors

Can the authors develop a generalization bound between the established bank and the new generated samples?

Comments on the paper

1 comment

Dear Reviewers, The authors have now submitted their responses to the reviews. There is only a limited amount of time left for you to review their replies and provide any further comments.

Your feedback is crucial to ensure that the review process proceeds smoothly. We would greatly appreciate it if you could share your reactions before the discussion period closes.

Thank you very much for your work. Best regards.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.