acceptodds
Under review as a conference paper at ICLR 2027

Faster Density Estimation and Inference-Time Steering for Flow Language Models

Abstract

Flow language models generate discrete data by numerical integration in a continuous space using a learned denoiser. In order to estimate log-densities, one typically needs to estimate the divergence of the learned network. This is often achieved using the Hutchinson's estimator, which incurs notable computational overhead. We propose an alternative density estimation mechanism, which we term as Flow-based Analytic TimE-score (FATE), that is consistent, has negligible computational overhead over denoiser evaluations, and unlocks applications of ODE simulations for general inference-time steering problems.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Flow language models, Density estimation, Inference-time steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yD5dOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper considers steering of pretrained flow language model at inference time for density estimation. The proposed FATE is a closed-form estimator of the log-density rate along the ODE. The estimator is then used in existing inference-time steering methods with ODE sampling: SuperDiff OR/AND (Skreta et al., 2025b) for composing models, and SMC with GLASS flows (Holderrieth et al., 2026) as proposal for tempering p(x)βp(x)^\beta and product of experts.

Critical strengths
  1. The core observation is simple and practical: since flow language models output the full categorical posterior, the posterior variance and therefore the divergence of the velocity can be computed in closed form without extra backward passes. This removes the stochastic probe noise of Hutchinson's estimator. The derivations are correct under an exact posterior.
  2. The efficiency gain over a reverse-mode Hutchinson implementation is clear: FATE costs about the same as a denoiser evaluation (Table 1), and the memory saving is large for CoBit models (Table 2).
Critical weaknesses
  1. Limited novelty, the contribution seems incremental:
  • The identity ∇x⋅mt=αtβt2 tr Cov[x1∣xt]\nabla_x \cdot m_t = \frac{\alpha_t}{\beta_t^2}\,\mathrm{tr}\,\mathrm{Cov}[x_1\mid x_t] is the second-order Tweedie formula (Meng et al., 2021), and the time score follows Yu et al. (2025) / Guth et al. (2025), as the paper itself acknowledges. The new part is the observation that tr Cov\mathrm{tr}\,\mathrm{Cov} is exact for categorical-output models.
  • The steering side is taken from existing work: off-trajectory density correction is from SuperDiff (Skreta et al., 2025b, Prop. 5–6); the tempering and product-of-experts targets are from Feynman-Kac Correctors (Skreta et al., 2025a); the ODE proposal is GLASS (Holderrieth et al., 2026); and the marginal density trick (Eq. 15) is from He et al. (2026). Overall I feel the paper is an application of these frameworks to flow language models with a cheaper divergence, and the two examples (SuperDiff, GLASS + SMC) do not make a strong case beyond that.
  • It is also not clear how much the ODE route is needed: SuperDiff's SDE version already tracks densities with the Itô density estimator without any divergence of the network, and FKC is also SDE-based. The argument that ODE sampling is more efficient is not tested on real flow language models.
  1. Lack of convincing evaluation on real models.
  • The experiments show feasibility but not an advantage over the baselines. The only setting where FATE is compared against Hutchinson inside a steering method is a synthetic task with exact denoisers. In the synthetic SMC (Fig. 4, Fig. 8), GLASS(FATE) and GLASS(Hutchinson) give similar sample quality (Final TV and Avg ESS overlap); FATE only improves the density error, which is expected with exact denoisers. No runtime is reported for this experiment, so the cost advantage is not shown in the steering setting either.
  • On real models, SMC composition collapses to about one unique text out of 1024 particles (KM and AS in Tables 5–7, e.g. 1.80±1.301.80 \pm 1.30 and 1.00±0.001.00 \pm 0.00). This is not discussed in the main text.
  1. Clarity / presentation issue. The current presentation of the paper is hard to follow.
  • The introduction stays at a high level (general LLM and discrete diffusion background), and the actual mechanism. But the main caveat is not clearly stated that the model outputs the full posterior, so the posterior variance and the divergence come for free.
  • Sec. 3.1 puts Algorithm 1 in without walking through it. The distinction between the reference velocity vtv_t and the evaluation velocity v~t\tilde v_t is not motivated, the derivation of step 2 is entirely deferred to App. B.1, and Remark 2 refers to inner GLASS flows before they are introduced.
  • The method name and Fig. 1 emphasize the time score ∂tlog⁡pt\partial_t \log p_t, but Algorithm 1 does not use it. It only uses Eq. 3 (which is exactly −∇⋅vt-\nabla\cdot v_t) and the Stein score. The time score is only an intermediate step in one of the proofs, which makes the main idea harder to see.
  • Sec. 3.2 (FATE+) introduces the coefficients cic_i without linking them to the SuperDiff weights kt(i)k_t^{(i)} in Eq. 10, and Sec. 4.2 has several notation errors (see the questions below), which makes the one new piece (how FATE supplies pτ∣τ−1p_{\tau|\tau-1} and pτp_\tau inside SMC) hard to follow.
  1. Baselines are limited. The real-model steering experiments only compare against the expert models themselves. Some suggested reasonable baselines would be:
  • SuperDiff with SDE sampling and the Itô density estimator (the same OR/AND task, no divergence needed), and score averaging (κ=0.5\kappa = 0.5), which is SuperDiff's own baseline;
  • FKC / reverse-SDE SMC with RNE densities (He et al., 2026) on real models;
  • GLASS(Hutchinson) on real models with wall-clock time at matched quality;
  • per-token logit temperature (Acquaviva, 2026; Deschenaux & Gulcehre, 2026) for tempering, which the paper mentions but does not compare to;
  • for density estimation: Hutchinson implemented with forward-mode JVP (which avoids storing activations and would likely shrink the 31.7× memory gap), Hutch++ (Meyer et al., 2021), the diffusion ELBO, and the distilled likelihood methods the paper cites (Ai et al., 2026; Rehman et al., 2026).
  1. Datasets and evaluation are narrow. Only two small model families are used, there is no ground-truth density check on real data, and the steering tasks do not include standard controlled-generation tasks or the reward alignment setting that motivates Sec. 2.

  2. Three closely related papers are not cited or discussed, which makes it hard to judge the positioning and novelty of the paper:

  • [1] shows that the posterior covariance Cov[x1∣xt]\mathrm{Cov}[x_1 \mid x_t] is an affine function of the Jacobian of the flow matching velocity, so tr Cov[x1∣xt]\mathrm{tr}\,\mathrm{Cov}[x_1\mid x_t] and ∇⋅vt\nabla\cdot v_t determine each other. This is the same identity FATE relies on (Eq. 7 / Eq. 87), stated for flow matching on continuous data.
  • [2] is a continuous embedding diffusion language model whose denoiser is the softmax-weighted average of token embeddings, i.e. the same categorical-posterior parameterization used by FATE, with the likelihood computed through a variational bound that does not need any divergence. It is both an earlier instance of the model class and an alternative way to obtain densities.
  • [3] includes SMC for temperature annealing and products of distributions with masked discrete diffusion language models, i.e. the same tempering and product-of-experts targets as Sec. 4.2 and Sec. 5.4–5.5, for a different family of language models.

[1] Xing, Jiarui, Song Wang, and Jian Wang. "Divergence is Uncertainty: A Closed-Form Posterior Covariance for Flow Matching." arXiv preprint arXiv:2605.00941 (2026). [2] Gulrajani, Ishaan, and Tatsunori B. Hashimoto. "Likelihood-based diffusion language models." Advances in Neural Information Processing Systems 36 (2023): 16693-16715. [3] Hasan, Mohsin, et al. "Discrete feynman-kac correctors." arXiv preprint arXiv:2601.10403 (2026).

Questions for authors
  • Beyond the closed-form divergence for categorical posteriors, which parts of Sec. 4 are new compared with Skreta et al. (2025a, 2025b), Holderrieth et al. (2026) and He et al. (2026)?
  • Why is ODE sampling needed here, given that SuperDiff's SDE version with the Itô density estimator does not require any divergence?
  • Could you compare ODE + FATE with SDE + Itô on the same composition tasks in terms of sample quality and wall-clock time?
  • How does FATE relate to the divergence–covariance identity in [1]? Beyond applying it to models with a categorical posterior (where tr Cov[x1∣xt]\mathrm{tr}\,\mathrm{Cov}[x_1\mid x_t] is exact), what is new in Eq. 3 and Prop. 3?
  • In Tables 5–7, SMC with KM and AS proposals ends up with 1.0–1.8 unique texts out of 1024 particles. Could you explain this collapse and how it supports contribution (iii)?
  • Could you report the end-to-end wall-clock time of a full steering run (SuperDiff or SMC) with FATE vs Hutchinson?
  • Could you consider adding standard controlled-generation tasks or the reward alignment setting from Sec. 2?

Notations:

  • Eq. 15 has log⁡pτ∣τ−1(xτ∣xτ−1)\log p_{\tau|\tau-1}(x_\tau \mid x_{\tau-1}) on both sides; should the left side be log⁡pτ−1∣τ(xτ−1∣xτ)\log p_{\tau-1|\tau}(x_{\tau-1} \mid x_\tau)?
  • In Sec. 3.1, nt(xt)=mt(xt)n_t(x_t) = m_t(x_t) equates a scalar with a vector; should it be nt=∑jmt(xt)jn_t = \sum_j m_t(x_t)_j?
5 comments
AuthorsOpenReview ↗

Response (part 1)

Thank you for your detailed reviews. Below we provide response organized by the weaknesses that you identified, and address your questions along with the response.

W 1. Limited novelty.

Q 1. which parts of Sec. 4 are new. Our main contribution is FATE, a closed-form estimator of the log-density rate for flow language models that can be evaluated essentially for free along sampling trajectories. For SuperDiff AND, [6] only derived the exact weights for the SDE case without considering ODEs, and we derived the formula for ODEs. We do not claim novelty beyond this for Sec. 4, and credit SuperDiff [6], SMC and the GLASS proposal [5] to the original works.

The positioning of this paper is to propose a methodological contribution to ODE-based inference time steering, and we do not claim that we are inventing a brand new inference time steering method. Our positioning is a methodological one: ODE-based inference-time steering is an active area of research [5] and so far for ODEs there was no better alternative than plugging in Hutchinson or GLASS flow density estimates into an inference-time sampler (e.g. SMC and SUperDiff), despite that ODE-based methods are generally more efficient than SDE-based methods [5]; this motivates adapting the ODE route instead of the SDE route. We isolate this density estimation component and provide an analytical estimator that can be evaluated for free along sampling trajectories so as to improve this component in terms of computational and memory efficiency. For empirical validation that ODE is more efficient than SDE under small numbers of function evaluations, please see our response to your W 4. and Q 2..

ODE-based sampling is an important sampling method for flow language models, a model family under active research. Tracking log densities along ODEs commonly relies on divergence estimators such Hutchinson's trace estimator, which can add substantial computational overhead. This cost motivated [6] to develop its density estimator to avoid Hutchinson's trace estimator along SDE trajectories. Our work is, to the best of our knowledge, the first to provide an estimator for a class of pretrained neural flow matching ODE that directly estimates the log-density rate from network outputs already computed during sampling. Our estimator does not require additional network evaluations, differentiating through the network or additional training, and hence is essentially free relative to ordinary ODE sampling. We believe such inexpensive density tracking is a necessary ingredient for efficient ODE-based inference-time steering of flow language models when the steering method requires intermediate density estimates.

ELBO density estimator While the ELBO-based density estimator [2, 7] allows evaluating ELBOs, with the standard diffusion sampler, using it for inference-time steering would lead to more function evaluations to estimate expectations over sampled noise levels, and one would still only obtain an ELBO bound instead of an estimate of the log density. These differences matter for inference-time steering applications.

W 2. Lack of convincing evaluation on real models

SMC + FATE does not improve much over SMC + GLASS in synthetic baselines. We agree that GLASS (FATE) does not improve upon GLASS (Hutchinson) in terms of performance. While GLASS (FATE) improves GLASS (Hutchinson) in terms of density estimation error, the improvements in other metrics are indeed less significiant. For this experiment we focus on validating the correctness of the resulting SMC algorithms using a known target distribution, and compare GLASS solutions against SDE solutions. We implement the denoiser analytically, and implement Hutchinson with random probes also analytically, hence the running time estimates may not be representative for neural models. We therefore address your Q 6. End-to-end wall-clock time of a full steering run later in this response, verifying that the overhead difference as shown e.g. in Figure 3 persists for downstream inference-time steering tasks on real models.

Q 5. particle degeneracy We entirely agree, and have updated the main text. Please also see Particle degeneracy in the response provided to all reviewers.

W 3. Clarity / presentation issue. Thank you for picking up on these, and we have fixed them. We removed Remark 2 and moved it later in the text. We added a comment on cic_i in FATE+ at the end of Section 4.1 where we discuss SuperDiff.

AuthorsOpenReview ↗

Response (part 2)

W 4. Baselines are limited We focused on demonstrating that our FATE estimator can be used inside the largely existing inference-time steering algorithms, and demonstrated that they indeed lead to reasonable steering. We do agree that it would benefit from comparing against some other baselines, and provide some results and discussion below.

SuperDiff with SDE sampling, Q 2. Why is ODE sampling needed here? and Q 3. Compare ODE+FATE with SDE+Itô on the same task You are right that SuperDiff SDE can perform sampling without Hutchinson. One motivation of GLASS flows [5] is that ODE sampling is significantly more efficient than SDE sampling, where ODE outperforms SDE at small NFEs. In order to validate whether this holds in our case, we benchmark ODE+FATE against SDE+RNE (with normal reference) [8], both integrated in the transformed coordinates, for SuperDiff Or; as shown in [8], RNE recovers Itô density estimator in continuous time, and with a reference, RNE outperforms Itô density estimator. This can be found in Appendix E.5 of the manuscript. We observe that our ODE+FATE notably outperforms SDE+RNE especially when the number of function evaluations is small. While the ODE variant is much more stable with fewer steps, maintaining mean GenPPL at 3.753.75 at 8 steps, SDE variant fails drastically, leading to mean GenPPL at 2780.992780.99 at 88 steps.

SuperDiff with score averaging Appendix E.5 also reports the results of using score averaging [15, 6] that you suggested. Overall, SuperDiff OR provides a much more faithful representation of the mixture of two models than directly using score averaging.

GLASS (Hutchinson) on real models and Q 6. End-to-end wall-clock time of a full steering run In our submitted manuscript, we compared FATE and Hutchinson both in terms of runtime and memory, where we observed that FATE is faster and more memory efficient. We now validate whether this translates to faster inference-time steering runs, reporting the running times of full steering runs of both SuperDiff and SMC using FATE, Hutchinson with 11 probe and Hutchinson with 1616 probes in Appendix E.4. When performing the steering experiments, we always use 10241024 particles, and use as batch size the maximum batch size that one NVIDIA GH200 can afford capped at 10241024, the number of samples for one SuperDiff run or one SMC run. FATE can often use a larger batch size than Hutchinson due to its smaller memory footprint. We observe that FATE consistently leads to notably smaller running times for full steering runs. For instance, for SuperDiff OR with CoBit-S LM1B on AG News, FATE takes around 42.442.4 seconds, compared with Hutchinson with one probe at around 117.8117.8 seconds, keeping the numbers of integration steps the same. While their running times differ much, the downstream sample qualities are largely the same, suggesting that FATE provides comparable accuracies while greatly reducing the computational costs.

Per-token logit temperature: While tempering per-token logits is indeed a useful way to interpolate between GenPPL and perplexities, it targets a different distribution than the tempered distribution that our tempering SMC considers. To the best of our knowledge, it is unclear what the target distribution of tempered logit is, since the score induced by tempered logits generally is not the same as the score of the stochastic interpolant induced by tempered data distribution.

For density estimation: forward mode Hutchinson: Previously, we adapted reverse-mode AD following LangFlow [12]. Now we additionally report experiments with forward mode Hutchinson. The FlashAttention kernel that we use with PyTorch does not support forward-mode AD through the network. As such, we instead use PyTorch math attention, which leads to different computational properties and generally slower running time and larger memory overheads for all methods. As you anticipated, forward mode Hutchinson indeed leads to smaller overhead in terms of memory.

Hutch++ reduces variance, but does not reduces the computational costs.

Diffusion ELBO: please see our discussion for your W 1., named ELBO density estimator.

Distilled likelihood methods: We believe these works on distilling likelihood complements our FATE, and in fact FATE can be used within them. Recent work [11] improves upon (Ai et al., 2026) by avoiding Hutchinson's trace estimator using the time score formula from Yu et al. (2025) / Guth et al. (2025) so as to improve efficiency and reduce variance. However, [11] still recovers a stochastic estimator, and in order to obtain the infinitesimal density estimate one would still need training apart from the original flow model. Instead, FATE is training-free and can be directly obtained while sampling the original model. Future work can explore using FATE inside SCALLOP for further variance reductions.

AuthorsOpenReview ↗

Response (part 3)

W 5. Datasets and evaluation are narrow. and Two small models At the same of submission, we used Langflow [12] and Cobit [13], which were among the larger open-source flow language models that achieved strong empirical performance and have code and model available to the public; both were accepted to NeurIPS 2026. We experimented with all five of their models: quoting their official descriptions on Hugging Face, LangFlow LM1B and OWT have ~130M parameters, CoBit-S LM1B and OWT are of size 130M and CoBit-M OWT is 462M. Plaid that you mentioned is indeed a highly relevant model. However, it did not achieve strong performance due to suboptimal neural network architecture [7]. Though versions of [7] appeared on arXiv before the ICLR submission deadline, we did not identify publicly available code or pretrained models of [7] at that time. Since then, even larger models have been proposed [9, 10], submitted to arXiv in October 2026 which is after the ICLR submission deadline. We gently remark that, as discussed in our manuscript, our FATE estimator is justified for all models falling under the framework of flow language models.

No ground-truth density check on real data. We focus on studying the existing models, and since these models are trained on LM1B [16] and OWT [17], we do not have access to the ground truth data density. We note that FATE's log-density rate estimates are, by definition, exact, for a trained model with no estimation error, as such when FATE does not give the ground truth data log-density rate, ignoring numerical issues, it is necessarily due to the model being imperfect.

Q 7. Standard controlled-generation tasks or the reward alignment settings We gently remark that the composition tasks that we considered are, to some extent, controlled-generation tasks, where the sampling method attempts to balance the two models, which are finetuned on different datasets and hence capture different texts. Reward-guided controlled-generation, i.e. reward alignment, is not a good application for our estimator, as existing inference-time steering methods can get by without density estimates; see e.g. [12]. We agree that the motivation in Sec. 2 can be improved, and have dropped the part specifically mentioning reward alignment.

W 6. Three closely related papers Thank you for pointing these out, and we entirely agree that they are relevant to our work and have cited them. Note that we cited [1] under its updated title, Uncertainty in a Single Pass: A Closed-Form Identity for One-Step Flow Matching, according to its v5 on arXiv.

[1], Q 4. How does FATE relate to the identity [1] The divergence-covariance identity in [1] is indeed relevant to FATE. However, [1] did not note that density estimates in the form of Eq. 3 could be obtained in closed-form for flow language models, and this is our new contribution. Prop. 3 as labeled in the original manuscript further shows that this enables tracking densities along the combination of multiple models; we note that due to editing, it is now labeled as Prop. 2 in the updated manuscript.

[2] The modeling family of [2] indeed falls under the framework of flow language model, and FATE should be applicable to it. We gently remark that we cited [7], which is a direct follow-up work of [2] and provides a more modern treatment. Please see our response for your W 1. named ELBO density estimator regarding the variational estimator as in [2, 7] compared with FATE and for your W 5. regarding why [2] is not included in our experiments.

[3] This paper focuses on discrete diffusion models building upon continuous-time Markov chain, which is a different family of diffusion models for text data from flow language models. As such, its proposed algorithms do not directly apply in our case.

Reviewer_bB6fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

FATE estimates log-density evolution in flow language models from posterior moments available in categorical or binary denoisers, avoiding velocity-network differentiation and stochastic trace estimation. The paper extends the estimator to modified denoisers and applies it to model composition and sequential Monte Carlo.

Critical strengths
  1. The paper identifies a useful connection between information already produced by a model and an otherwise expensive density calculation. This provides a conceptually clear and potentially reusable route to more efficient probabilistic control of flow models.
  2. A useful structural observation that reuses existing model outputs. The estimator exploits posterior first and second moments already recoverable from token or bit probabilities. Under an exact denoiser, the density-rate identity is mathematically justified, and no additional model needs to be trained.
Critical weaknesses

1. Accuracy of the density ratios required by steering is not established for learned denoisers.
Remark 1 correctly acknowledges possible bias with imperfect denoisers. However, Figure 2 mainly establishes NLL correlation, while §5.3 uses analytic denoisers. Neither establishes the accuracy of the exponentiated density ratios used by SMC or model composition. The missing validation concerns the estimator’s central practical use, rather than the correctness of its idealized identity.

2. Severe particle duplication undermines the evidence for practical target-distribution sampling.
Table 4 reports only 13.2 unique texts from 1,024 particles for CoBit-S OWT at T=0.95T=0.95, and composition results in Tables 5–7 often retain approximately one or two unique texts. Low perplexity under this degree of duplication is insufficient evidence of faithful target sampling. The paper does not adequately separate density-estimation error, proposal mismatch, and genuine target concentration.

Questions for authors
  1. On fixed trajectories of a learned model, how closely do FATE’s temporal log-density ratios and normalized importance weights agree with a numerically converged divergence-based reference?
  2. Holding proposals and sampling schedules fixed, does the duplication in Tables 4–7 persist when FATE density estimates are replaced by independently validated estimates?
  3. What are the pre-resampling ESS and surviving-ancestor counts for these experiments, and what evidence supports interpreting the low diversity as target concentration rather than particle degeneracy?
2 comments

Thank you for your thoughtful review.

Accuracy of the density ratios: We agree that this needs to be established. In the original submitted version, we visualized the densities of individual data points estimated using Hutchinson and FATE; the points largely lie around the diagonal, suggesting that the densities agree well. We now further investigate how accurate are the resulting densitiy ratios. As discussed in Density ratios along fixed trajectories of the response to all reviewers, we provided measures of the resulting density ratios, and analyzed whether and how these errors influence the downstream SMC weights. In general, using FATE and using Hutchinson do not lead to meaningfully different inference-time steering algorithms.

As discussed in Which density does FATE and Hutchinson track? of the response to all reviewers, even though FATE and Hutchinson differ for models with estimation errors and trained models almost always have such errors, FATE could still provide meaningful approximations of the data density despite differing from Hutchinson, and the assumption of perfect model that FATE builds upon is similar to previous works.

severe particle duplication: We provide discussion in Particle degeneracy of the response to all reviewers. The degeneracy issues generally persist when replacing FATE with Hutchinson with potentially as many as 1616 probes, suggesting that this is likely inherent to SMC.

Pre-resampling ESS and surviving-ancestor counts: We provided such statistics for the SMC experiments that suffered from notable degeneracy in Appendix E.4. Both the minimum pre-resampling ESS and the surviving-ancestor counts remain low, and we interpret this as particle degeneracy indeed occurred. The numbers of unique texts are notably higher when the temperature is closer to 11 in SMC for tempering, hinting that the degeneracy is likely caused by both the high dimensionality and the lack of an ideal proposal. For SMC for tempering, GenPPL and entropy already interpolate across wide ranges even when temperature slightly deviates from 11. For composition, we recommend SuperDiff as the more practical solution, which is why we focus on SuperDiff in the main text.

Thank you for the detailed clarifications and additional experiments. While I appreciate the authors' efforts to address my comments, my primary concern remains that the methodological novelty is relatively incremental and its applicability is somewhat limited. Therefore, I will maintain my original rating.

Reviewer_z7opOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FATE, an analytic density estimator for flow language models. It uses conditional moments available from a model's token probabilities. These moments replace a costly stochastic estimate of the velocity divergence. The formula is exact when the model gives the correct conditional distributions.

The paper also gives extensions for combinations of models and tempered token probabilities. Experiments compare density estimates, execution time, and memory use. Further experiments use the estimator for model composition and sampling from tempered distributions.

Critical strengths

The paper addresses a concrete problem. Density estimation often requires derivatives through a large network. FATE uses information already available from the denoiser. It can therefore help existing models without retraining.

The practical insight has value despite the simple derivation. The paper connects established score identities with the conditional moments available in flow language models. This connection gives an implementable estimator with little extra computation. A new steering algorithm is not necessary for this contribution to be useful.

The measured savings are substantial. The reported speed improvement reaches about three times, with large memory reductions in some models. Experiments include LangFlow and CoBit models, as well as synthetic distributions with known answers. This evidence supports the estimator as a useful computational tool.

Critical weaknesses

High correlation does not establish density accuracy. For an imperfect denoiser, FATE can differ systematically from the density of the learned flow. Remark 1 correctly states this limitation. The comparison focuses on whether FATE and Hutchinson estimates follow the same pattern. However, their estimated log-density values can still differ consistently. High correlation does not show whether FATE estimates the density ratios accurately. The paper should measure errors in these ratios because they determine the importance weights used for steering.

The application evidence is weaker than the efficiency evidence. Tables 5–7 show that several SMC composition settings produce only one or two distinct texts, which indicates low sample diversity. SuperDiff AND also does not consistently increase the proportion of texts in the shared category. Equal density under both models does not guarantee that a text satisfies both target conditions. These outcomes limit the claim of effective steering, although they do not remove the value of faster density estimates.

Questions for authors

How large are FATE errors in the density ratios used for steering, compared with a high-precision reference with many Hutchinson probes? At matched sampling cost, how does FATE affect target agreement and sample diversity compared with Hutchinson-based steering?

1 comment

Thank you for highlighting the usefulness of FATE.

The estimated log-densities using FATE vs. Hutchinson can differ even if the correlation is high. We agree and that is why we plotted all data points used to compute the correlation in Figure 2 and Figure 7 in the original submission: because they all lie on the diagonal, this suggests that the log-densities do not differ much. However, the reviewer suggested that while the density evaluations may agree, the density ratios may not. We agree and discuss this at length in the paragraph Density ratios along fixed trajectories for all reviewers. The response to all reviewers additionally provide a paragraph on Which density does FATE and Hutchinson track?, discussing what it means for FATE and Hutchinson to differ under imperfect models, and highlighting that the perfect model assumption that FATE uses is similar to existing works like FKC and RNE.

The application evidence is weaker than the efficiency evidence: We provided a discussion on sample diversity in Particle degeneracy paragraph in the response to all reviewers.

We also agree with your observation about SuperDiff AND. It simply implies that two models would assign equal densities to the points, without directly implying the category proportions.

How does FATE affect target agreement and sample diversity compared with Hutchinson-based steering? We provided such a study in Appendix E.4, where we control for the total number of particles and the total number of sampling steps; note that FATE is faster to run than Hutchinson. The behaviors of SuperDiff and SMC are generally consistent when using FATE or Hutchinson.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to all reviewers (part 1)

Thank you for your careful reviews. We have revised the manuscript, marked the changes in blue, and added Figure 10, Figure 11 and Figure 12. Several reviewers raised two questions about steering: (1) how well do FATE and Hutchinson agree on the density ratios that determine SMC weights, and (2) why do some runs result in only one or two distinct texts (degeneracy). We provided new theoretical analysis and added two experiments to answer this. We find that the normalized density ratios and importance weights are largely stable for FATE and Hutchinson, despite the raw density ratios may vary more. We also find that severe particle degeneracy persists when using Hutchinson to estimate the densities, even when using 1616 probes.

Which density does FATE and Hutchinson track? We distinguish between three values of the log-density rate dtlog⁡pt(xt)d_t\log p_t(x_t):

  • case 1: are approximated using the FATE estimator.
  • case 2: are approximated uisng the Hutchinson's trace estimator.
  • case 3: correspond to the log-density rate of the noisy-data density induced by the stochastic interpolant.

The first 2 can be estimated using the model, while the last is generally intractable.

Appendix E.1 presents new theoretical analysis on the behaviors of the two estimators when the denoiser incurs some estimation error. While Hutchinson provides an unbiased estimator of the divergence, FATE uses the predicted posterior moments to estimate the log-density change. The difference between FATE and data log-density rate can be bounded in terms of the errors of the predicted first and second moments, which can in turn be bounded using total variation or cross entropy between the estimated component-wise probabilities and the ones determined by the stochastic interpolant. The difference between Hutchinson and FATE and the difference between Hutchinson and data density instead also depends on the derivative of the predicted denoiser. A model can approximate the posterior moments reasonably well while having worse derivative estimates, as the training loss generally does not directly train the derivatives, in which case FATE can be a better approximation of the data density than Hutchinson.

FATE is defined in terms of the estimated component-wise probabilities: whenever they have no estimation error, then FATE is actually equal to the total log-density rate dtlog⁡pt(xt)d_t\log p_t(x_t) of the data density. Previous works on density estimation along SDE trajectories rely on similar assumptions [1, 2, 3]. Specifically, the exactness of Itô density estimator as in [1] and FKC as in [2] relies on the assumption that the learned scores are perfect, and the correctness of the pathwise identity of RNE in [3] relies on the assumption that the learned denoising process is the exact time-reversal of the noising process. All of these do not hold in general when the model is not perfectly trained, similar to FATE. We have included such a discussion in the paper.

Nevertheless, we entirely agree that the differences between FATE and Hutchinson should be quantified on real models, and hence performed experiments to evaluate them.

Density ratios along fixed trajectories

In the submitted version of this paper, we showed that the FATE and Hutchinson lead to similar density estimates. We gently remark that, in Figure 2 and Figure 7, apart from reporting the correlations, we also provided the scatter plots of the individual densities. These density points generally lie close to the diagonal, suggesting that FATE and Hutchinson lead to similar density estimates for the data points. Here, we further evaluate whether the FATE and Hutchinson estimators lead to similar density ratio estimates along the trajectories. In the following, we broadly show that FATE and Hutchinson generally lead to similar normalized density ratios and SMC weights, but potentially different density ratios. This of course is more nuanced, and we explain the nuance in the following.

In Appendix E.3, we quantify the differences between the density ratios estimated using FATE and Hutchinson with 1616 Gaussian probes. We consider two scenarios: (1) compare the density ratios along ordinary ODE generation trajectories, which provide the density estimates for algorithms like SuperDiff, (2) compare the density ratios along SMC trajectories for tempering and composition.

AuthorsOpenReview ↗

Response to all reviewers (part 2)

For (1), apart from considering the errors of the raw density ratios, we additionally consider the errors of centered density ratios. Centered density ratios rely on the assumption that FATE and Hutchinson differ by some constant that only depends on tt; this is meaningful, because practical inference-time steering algorithms often perform self-normaliziation, removing such constants. For LangFlow models, the centered RMSE are 5.465.46 and 20.0720.07, comparable to RMSE between two Hutchinson estimators with 88 probes at 6.246.24 and 20.0620.06. The centered RMSEs are however larger for CoBit models, and the raw density ratios for the models are still larger, though does not lead to large differences in downstream inference-time steering algorithms, as verified in Appendix E.4.

For (2), we report the log ratio MSE and the differences in the resulting SMC weights for SMC tempering and composition. The differences between the resulting weights as reflected by mean weight TV remain controlled, though log-ratio MSE and maximum weight TV may be larger.

SettingLog-ratio MSE (nats²)Mean weight TVMaximum weight TV
CoBit-S OWT tempering, T=0.95T=0.95306.8160.02610.2193
CoBit-S LM1B / AG News KM0.833 / 0.7700.01550.1561
CoBit-S LM1B / AG News AS0.667 / 0.6780.00660.0432

Particle degeneracy In Appendix E.4, in order to validate whether using FATE instead of Hutchinson leads to the particle degeneracy observed in some SMC runs, we let FATE, H1 and H16 each drive their respective SMC sampling on cases where SMC were observed to lead to certain degenracy. In general, switching from FATE to Hutchinson does not eliminate particle degeneracy in the tested settings.

We attribute the degeneracy to the difficulty of performing SMC inference on a large model with a proposal that might not be ideal, consistent with previous SMC literature [4, 5]. The degeneracy is especially obvious for composition SMC, in which case we recommend using SuperDiff. We updated the manuscript accordingly.

[1] The Superposition of Diffusion Models Using the Itô Density Estimator, Skreta et al., ICLR 2025

[2] Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts, Skreta et al., ICML 2025

[3] RNE: plug-and-play diffusion inference-time control and energy-based training, He et al., ICLR 2026

[4] Sharp failure rates for the bootstrap particle filter in high dimensions, Bickel et al., IMS Collections: Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 2008

[5] Elements of Sequential Monte Carlo, Naesseth et al., Foundations and Trends@ in Machine Learning, 2019

Area_Chair_74qpOpenReview ↗

Author-Reviewer Discussion ends in 1 day, 12 hours

Dear authors and reviewers,

Thank you to the authors for their detailed responses and the revised manuscript of Nov 16, with changes marked in blue. The discussion period closes in about a day and a half, so this is the last chance for a direct exchange before the reviewer-AC discussion starts on Nov 19.

Reviewers: I haven’t yet seen a reply from any of you. Please read the authors’ responses and the revision, which address the main points raised in your reviews. Please let us know whether your concerns have been addressed, and update your recommendation if your assessment has changed. Even a short acknowledgment helps.

Authors: please be ready to answer any follow-up questions in the time left, focusing on the points most likely to affect the decision.

Thank you all for keeping the discussion constructive and timely.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.