- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper considers steering of pretrained flow language model at inference time for density estimation. The proposed FATE is a closed-form estimator of the log-density rate along the ODE. The estimator is then used in existing inference-time steering methods with ODE sampling: SuperDiff OR/AND (Skreta et al., 2025b) for composing models, and SMC with GLASS flows (Holderrieth et al., 2026) as proposal for tempering and product of experts.
Critical strengths
- The core observation is simple and practical: since flow language models output the full categorical posterior, the posterior variance and therefore the divergence of the velocity can be computed in closed form without extra backward passes. This removes the stochastic probe noise of Hutchinson's estimator. The derivations are correct under an exact posterior.
- The efficiency gain over a reverse-mode Hutchinson implementation is clear: FATE costs about the same as a denoiser evaluation (Table 1), and the memory saving is large for CoBit models (Table 2).
Critical weaknesses
- Limited novelty, the contribution seems incremental:
- The identity is the second-order Tweedie formula (Meng et al., 2021), and the time score follows Yu et al. (2025) / Guth et al. (2025), as the paper itself acknowledges. The new part is the observation that is exact for categorical-output models.
- The steering side is taken from existing work: off-trajectory density correction is from SuperDiff (Skreta et al., 2025b, Prop. 5–6); the tempering and product-of-experts targets are from Feynman-Kac Correctors (Skreta et al., 2025a); the ODE proposal is GLASS (Holderrieth et al., 2026); and the marginal density trick (Eq. 15) is from He et al. (2026). Overall I feel the paper is an application of these frameworks to flow language models with a cheaper divergence, and the two examples (SuperDiff, GLASS + SMC) do not make a strong case beyond that.
- It is also not clear how much the ODE route is needed: SuperDiff's SDE version already tracks densities with the Itô density estimator without any divergence of the network, and FKC is also SDE-based. The argument that ODE sampling is more efficient is not tested on real flow language models.
- Lack of convincing evaluation on real models.
- The experiments show feasibility but not an advantage over the baselines. The only setting where FATE is compared against Hutchinson inside a steering method is a synthetic task with exact denoisers. In the synthetic SMC (Fig. 4, Fig. 8), GLASS(FATE) and GLASS(Hutchinson) give similar sample quality (Final TV and Avg ESS overlap); FATE only improves the density error, which is expected with exact denoisers. No runtime is reported for this experiment, so the cost advantage is not shown in the steering setting either.
- On real models, SMC composition collapses to about one unique text out of 1024 particles (KM and AS in Tables 5–7, e.g. and ). This is not discussed in the main text.
- Clarity / presentation issue. The current presentation of the paper is hard to follow.
- The introduction stays at a high level (general LLM and discrete diffusion background), and the actual mechanism. But the main caveat is not clearly stated that the model outputs the full posterior, so the posterior variance and the divergence come for free.
- Sec. 3.1 puts Algorithm 1 in without walking through it. The distinction between the reference velocity and the evaluation velocity is not motivated, the derivation of step 2 is entirely deferred to App. B.1, and Remark 2 refers to inner GLASS flows before they are introduced.
- The method name and Fig. 1 emphasize the time score , but Algorithm 1 does not use it. It only uses Eq. 3 (which is exactly ) and the Stein score. The time score is only an intermediate step in one of the proofs, which makes the main idea harder to see.
- Sec. 3.2 (FATE+) introduces the coefficients without linking them to the SuperDiff weights in Eq. 10, and Sec. 4.2 has several notation errors (see the questions below), which makes the one new piece (how FATE supplies and inside SMC) hard to follow.
- Baselines are limited. The real-model steering experiments only compare against the expert models themselves. Some suggested reasonable baselines would be:
- SuperDiff with SDE sampling and the Itô density estimator (the same OR/AND task, no divergence needed), and score averaging (), which is SuperDiff's own baseline;
- FKC / reverse-SDE SMC with RNE densities (He et al., 2026) on real models;
- GLASS(Hutchinson) on real models with wall-clock time at matched quality;
- per-token logit temperature (Acquaviva, 2026; Deschenaux & Gulcehre, 2026) for tempering, which the paper mentions but does not compare to;
- for density estimation: Hutchinson implemented with forward-mode JVP (which avoids storing activations and would likely shrink the 31.7× memory gap), Hutch++ (Meyer et al., 2021), the diffusion ELBO, and the distilled likelihood methods the paper cites (Ai et al., 2026; Rehman et al., 2026).
-
Datasets and evaluation are narrow. Only two small model families are used, there is no ground-truth density check on real data, and the steering tasks do not include standard controlled-generation tasks or the reward alignment setting that motivates Sec. 2.
-
Three closely related papers are not cited or discussed, which makes it hard to judge the positioning and novelty of the paper:
- [1] shows that the posterior covariance is an affine function of the Jacobian of the flow matching velocity, so and determine each other. This is the same identity FATE relies on (Eq. 7 / Eq. 87), stated for flow matching on continuous data.
- [2] is a continuous embedding diffusion language model whose denoiser is the softmax-weighted average of token embeddings, i.e. the same categorical-posterior parameterization used by FATE, with the likelihood computed through a variational bound that does not need any divergence. It is both an earlier instance of the model class and an alternative way to obtain densities.
- [3] includes SMC for temperature annealing and products of distributions with masked discrete diffusion language models, i.e. the same tempering and product-of-experts targets as Sec. 4.2 and Sec. 5.4–5.5, for a different family of language models.
[1] Xing, Jiarui, Song Wang, and Jian Wang. "Divergence is Uncertainty: A Closed-Form Posterior Covariance for Flow Matching." arXiv preprint arXiv:2605.00941 (2026). [2] Gulrajani, Ishaan, and Tatsunori B. Hashimoto. "Likelihood-based diffusion language models." Advances in Neural Information Processing Systems 36 (2023): 16693-16715. [3] Hasan, Mohsin, et al. "Discrete feynman-kac correctors." arXiv preprint arXiv:2601.10403 (2026).
Questions for authors
- Beyond the closed-form divergence for categorical posteriors, which parts of Sec. 4 are new compared with Skreta et al. (2025a, 2025b), Holderrieth et al. (2026) and He et al. (2026)?
- Why is ODE sampling needed here, given that SuperDiff's SDE version with the Itô density estimator does not require any divergence?
- Could you compare ODE + FATE with SDE + Itô on the same composition tasks in terms of sample quality and wall-clock time?
- How does FATE relate to the divergence–covariance identity in [1]? Beyond applying it to models with a categorical posterior (where is exact), what is new in Eq. 3 and Prop. 3?
- In Tables 5–7, SMC with KM and AS proposals ends up with 1.0–1.8 unique texts out of 1024 particles. Could you explain this collapse and how it supports contribution (iii)?
- Could you report the end-to-end wall-clock time of a full steering run (SuperDiff or SMC) with FATE vs Hutchinson?
- Could you consider adding standard controlled-generation tasks or the reward alignment setting from Sec. 2?
Notations:
- Eq. 15 has on both sides; should the left side be ?
- In Sec. 3.1, equates a scalar with a vector; should it be ?
5 comments
Response (part 2)
W 4. Baselines are limited We focused on demonstrating that our FATE estimator can be used inside the largely existing inference-time steering algorithms, and demonstrated that they indeed lead to reasonable steering. We do agree that it would benefit from comparing against some other baselines, and provide some results and discussion below.
SuperDiff with SDE sampling, Q 2. Why is ODE sampling needed here? and Q 3. Compare ODE+FATE with SDE+Itô on the same task You are right that SuperDiff SDE can perform sampling without Hutchinson. One motivation of GLASS flows [5] is that ODE sampling is significantly more efficient than SDE sampling, where ODE outperforms SDE at small NFEs. In order to validate whether this holds in our case, we benchmark ODE+FATE against SDE+RNE (with normal reference) [8], both integrated in the transformed coordinates, for SuperDiff Or; as shown in [8], RNE recovers Itô density estimator in continuous time, and with a reference, RNE outperforms Itô density estimator. This can be found in Appendix E.5 of the manuscript. We observe that our ODE+FATE notably outperforms SDE+RNE especially when the number of function evaluations is small. While the ODE variant is much more stable with fewer steps, maintaining mean GenPPL at at 8 steps, SDE variant fails drastically, leading to mean GenPPL at at steps.
SuperDiff with score averaging Appendix E.5 also reports the results of using score averaging [15, 6] that you suggested. Overall, SuperDiff OR provides a much more faithful representation of the mixture of two models than directly using score averaging.
GLASS (Hutchinson) on real models and Q 6. End-to-end wall-clock time of a full steering run In our submitted manuscript, we compared FATE and Hutchinson both in terms of runtime and memory, where we observed that FATE is faster and more memory efficient. We now validate whether this translates to faster inference-time steering runs, reporting the running times of full steering runs of both SuperDiff and SMC using FATE, Hutchinson with probe and Hutchinson with probes in Appendix E.4. When performing the steering experiments, we always use particles, and use as batch size the maximum batch size that one NVIDIA GH200 can afford capped at , the number of samples for one SuperDiff run or one SMC run. FATE can often use a larger batch size than Hutchinson due to its smaller memory footprint. We observe that FATE consistently leads to notably smaller running times for full steering runs. For instance, for SuperDiff OR with CoBit-S LM1B on AG News, FATE takes around seconds, compared with Hutchinson with one probe at around seconds, keeping the numbers of integration steps the same. While their running times differ much, the downstream sample qualities are largely the same, suggesting that FATE provides comparable accuracies while greatly reducing the computational costs.
Per-token logit temperature: While tempering per-token logits is indeed a useful way to interpolate between GenPPL and perplexities, it targets a different distribution than the tempered distribution that our tempering SMC considers. To the best of our knowledge, it is unclear what the target distribution of tempered logit is, since the score induced by tempered logits generally is not the same as the score of the stochastic interpolant induced by tempered data distribution.
For density estimation: forward mode Hutchinson: Previously, we adapted reverse-mode AD following LangFlow [12]. Now we additionally report experiments with forward mode Hutchinson. The FlashAttention kernel that we use with PyTorch does not support forward-mode AD through the network. As such, we instead use PyTorch math attention, which leads to different computational properties and generally slower running time and larger memory overheads for all methods. As you anticipated, forward mode Hutchinson indeed leads to smaller overhead in terms of memory.
Hutch++ reduces variance, but does not reduces the computational costs.
Diffusion ELBO: please see our discussion for your W 1., named ELBO density estimator.
Distilled likelihood methods: We believe these works on distilling likelihood complements our FATE, and in fact FATE can be used within them. Recent work [11] improves upon (Ai et al., 2026) by avoiding Hutchinson's trace estimator using the time score formula from Yu et al. (2025) / Guth et al. (2025) so as to improve efficiency and reduce variance. However, [11] still recovers a stochastic estimator, and in order to obtain the infinitesimal density estimate one would still need training apart from the original flow model. Instead, FATE is training-free and can be directly obtained while sampling the original model. Future work can explore using FATE inside SCALLOP for further variance reductions.
Response (part 3)
W 5. Datasets and evaluation are narrow. and Two small models At the same of submission, we used Langflow [12] and Cobit [13], which were among the larger open-source flow language models that achieved strong empirical performance and have code and model available to the public; both were accepted to NeurIPS 2026. We experimented with all five of their models: quoting their official descriptions on Hugging Face, LangFlow LM1B and OWT have ~130M parameters, CoBit-S LM1B and OWT are of size 130M and CoBit-M OWT is 462M. Plaid that you mentioned is indeed a highly relevant model. However, it did not achieve strong performance due to suboptimal neural network architecture [7]. Though versions of [7] appeared on arXiv before the ICLR submission deadline, we did not identify publicly available code or pretrained models of [7] at that time. Since then, even larger models have been proposed [9, 10], submitted to arXiv in October 2026 which is after the ICLR submission deadline. We gently remark that, as discussed in our manuscript, our FATE estimator is justified for all models falling under the framework of flow language models.
No ground-truth density check on real data. We focus on studying the existing models, and since these models are trained on LM1B [16] and OWT [17], we do not have access to the ground truth data density. We note that FATE's log-density rate estimates are, by definition, exact, for a trained model with no estimation error, as such when FATE does not give the ground truth data log-density rate, ignoring numerical issues, it is necessarily due to the model being imperfect.
Q 7. Standard controlled-generation tasks or the reward alignment settings We gently remark that the composition tasks that we considered are, to some extent, controlled-generation tasks, where the sampling method attempts to balance the two models, which are finetuned on different datasets and hence capture different texts. Reward-guided controlled-generation, i.e. reward alignment, is not a good application for our estimator, as existing inference-time steering methods can get by without density estimates; see e.g. [12]. We agree that the motivation in Sec. 2 can be improved, and have dropped the part specifically mentioning reward alignment.
W 6. Three closely related papers Thank you for pointing these out, and we entirely agree that they are relevant to our work and have cited them. Note that we cited [1] under its updated title, Uncertainty in a Single Pass: A Closed-Form Identity for One-Step Flow Matching, according to its v5 on arXiv.
[1], Q 4. How does FATE relate to the identity [1] The divergence-covariance identity in [1] is indeed relevant to FATE. However, [1] did not note that density estimates in the form of Eq. 3 could be obtained in closed-form for flow language models, and this is our new contribution. Prop. 3 as labeled in the original manuscript further shows that this enables tracking densities along the combination of multiple models; we note that due to editing, it is now labeled as Prop. 2 in the updated manuscript.
[2] The modeling family of [2] indeed falls under the framework of flow language model, and FATE should be applicable to it. We gently remark that we cited [7], which is a direct follow-up work of [2] and provides a more modern treatment. Please see our response for your W 1. named ELBO density estimator regarding the variational estimator as in [2, 7] compared with FATE and for your W 5. regarding why [2] is not included in our experiments.
[3] This paper focuses on discrete diffusion models building upon continuous-time Markov chain, which is a different family of diffusion models for text data from flow language models. As such, its proposed algorithms do not directly apply in our case.
Response (part 1)
Thank you for your detailed reviews. Below we provide response organized by the weaknesses that you identified, and address your questions along with the response.
W 1. Limited novelty.
Q 1. which parts of Sec. 4 are new. Our main contribution is FATE, a closed-form estimator of the log-density rate for flow language models that can be evaluated essentially for free along sampling trajectories. For SuperDiff AND, [6] only derived the exact weights for the SDE case without considering ODEs, and we derived the formula for ODEs. We do not claim novelty beyond this for Sec. 4, and credit SuperDiff [6], SMC and the GLASS proposal [5] to the original works.
The positioning of this paper is to propose a methodological contribution to ODE-based inference time steering, and we do not claim that we are inventing a brand new inference time steering method. Our positioning is a methodological one: ODE-based inference-time steering is an active area of research [5] and so far for ODEs there was no better alternative than plugging in Hutchinson or GLASS flow density estimates into an inference-time sampler (e.g. SMC and SUperDiff), despite that ODE-based methods are generally more efficient than SDE-based methods [5]; this motivates adapting the ODE route instead of the SDE route. We isolate this density estimation component and provide an analytical estimator that can be evaluated for free along sampling trajectories so as to improve this component in terms of computational and memory efficiency. For empirical validation that ODE is more efficient than SDE under small numbers of function evaluations, please see our response to your W 4. and Q 2..
ODE-based sampling is an important sampling method for flow language models, a model family under active research. Tracking log densities along ODEs commonly relies on divergence estimators such Hutchinson's trace estimator, which can add substantial computational overhead. This cost motivated [6] to develop its density estimator to avoid Hutchinson's trace estimator along SDE trajectories. Our work is, to the best of our knowledge, the first to provide an estimator for a class of pretrained neural flow matching ODE that directly estimates the log-density rate from network outputs already computed during sampling. Our estimator does not require additional network evaluations, differentiating through the network or additional training, and hence is essentially free relative to ordinary ODE sampling. We believe such inexpensive density tracking is a necessary ingredient for efficient ODE-based inference-time steering of flow language models when the steering method requires intermediate density estimates.
ELBO density estimator While the ELBO-based density estimator [2, 7] allows evaluating ELBOs, with the standard diffusion sampler, using it for inference-time steering would lead to more function evaluations to estimate expectations over sampled noise levels, and one would still only obtain an ELBO bound instead of an estimate of the log density. These differences matter for inference-time steering applications.
W 2. Lack of convincing evaluation on real models
SMC + FATE does not improve much over SMC + GLASS in synthetic baselines. We agree that GLASS (FATE) does not improve upon GLASS (Hutchinson) in terms of performance. While GLASS (FATE) improves GLASS (Hutchinson) in terms of density estimation error, the improvements in other metrics are indeed less significiant. For this experiment we focus on validating the correctness of the resulting SMC algorithms using a known target distribution, and compare GLASS solutions against SDE solutions. We implement the denoiser analytically, and implement Hutchinson with random probes also analytically, hence the running time estimates may not be representative for neural models. We therefore address your Q 6. End-to-end wall-clock time of a full steering run later in this response, verifying that the overhead difference as shown e.g. in Figure 3 persists for downstream inference-time steering tasks on real models.
Q 5. particle degeneracy We entirely agree, and have updated the main text. Please also see Particle degeneracy in the response provided to all reviewers.
W 3. Clarity / presentation issue. Thank you for picking up on these, and we have fixed them. We removed Remark 2 and moved it later in the text. We added a comment on ci in FATE+ at the end of Section 4.1 where we discuss SuperDiff.