acceptodds
Under review as a conference paper at ICLR 2027

BCT: Bayes-Consistent Tempering for Flow Language Models

Abstract

Temperature scaling is typically applied as a post-hoc transformation of model probabilities. In posterior-induced flow language models, this creates two mismatches. First, posterior tempering implicitly changes both the endpoint distribution and the distribution of intermediate noisy states, while the denoiser remains trained on the original states. Second, the tempered posterior weights are combined with transport dynamics derived for the untempered flow. These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes-Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow. Its learned realization trains the denoiser on tempered intermediate states and tempers the endpoint distribution using the corresponding tempered dynamics. We also derive training-free variants for conventionally trained checkpoints. Across hyperspherical flow LMs with two denoiser architectures and a von Mises–Fisher flow LM, BCT improves GSM8K generation across broad temperature ranges and achieves the strongest tempering performance. The gains also extend to Sudoku, demonstrating benefits on tightly constrained generation. At very low temperatures, the learned realization is also markedly more robust than the training-free variants, supporting temperature as a property of the generative flow across training and inference rather than a post-hoc probability adjustment.

Primary area:
generative models
Keywords:
flow language models, sampling temperature, tempering, non-autoregressive language generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_f2GvOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines temperature scaling in posterior-induced flow language models. The authors point out that standard posterior tempering can create a mismatch between the tempered distribution and the underlying sampling dynamics. To address this, they propose Bayes-Consistent Tempering (BCT), which adjusts the intermediate distributions and conditional velocities to account for the temperature change. The paper develops both training-based and training-free approaches for hyperspherical and von Mises–Fisher flows. Experiments on GSM8K and Sudoku show that BCT generally performs better than standard tempering, with the most noticeable improvements at lower temperatures.

Critical strengths

1.The method is well motivated. The authors identify a concrete inconsistency in standard posterior tempering and derive bridge-aware conditional dynamics for two distinct flow constructions. This provides a useful theoretical perspective on temperature control in flow language models.

2.The experimental comparisons are informative. The paper includes matched continuation baselines, training-free variants, two denoiser backbones, and Pass@K evaluations. The improvements on GSM8K are substantial, and the Sudoku results provide additional evidence beyond mathematical reasoning.

Critical weaknesses

1.The main implementation does not fully preserve the claimed Bayesian consistency. Equation (8) gives the exact correction, but Equation (9) introduces a tunable coefficient λ. The main GSM8K results use λ = 0.5 for S-arch and λ = 0.25 for DiT, rather than λ = 1. For these values, the resulting posterior generally no longer corresponds to the tempered bridge whose conditional velocity is used. This weakens the connection between the theoretical motivation and the reported improvements.

2.The empirical scope remains limited. The evaluation focuses on GSM8K and Sudoku, with only one training run per continuation. The reported uncertainty therefore does not capture training variability. The vMF experiments also show a noticeable performance drop for BCT-L at T = 1. These results make the broader robustness claim less convincing.

Questions for authors

1.Can the authors report GSM8K results with the exact Bayes correction (λ = 1) and explain how the selected partial corrections remain consistent with the sampling dynamics?

2.Can the authors separate the effects of tempered-bridge training, corrected conditional velocities, and the tuned posterior correction? How sensitive are the gains to training seeds?

3 comments
AuthorsOpenReview ↗

Response to Reviewer f2Gv

We thank the reviewer for the careful reading. In short: the exact correction (λ=1\lambda=1) already gives the gains in three of four settings, and λ<1\lambda<1 is a principled safeguard against denoiser error; the corrected velocity carries most of the gain, with TFT keeping it under strong tempering; the gains hold across training seeds; and the vMF drop at T=1T=1 reflects the checkpoint, not Proposition 1. All numbers are GSM8K test accuracy (%).


W1 / Q1: Exact correction (λ=1\lambda=1) and the role of λ\lambda

Exactness holds only at λ=1\lambda=1, and λ=1\lambda=1 alone already gives the gains in three of four settings. On S-arch, λ=1\lambda=1 outperforms standard tempering at every T<1T<1; on Sudoku and the vMF flow, the held-out selection already chose λ=1\lambda=1. On DiT, λ=1\lambda=1 degrades under strong tempering (3 generation seeds):

standard / λ=1\lambda=1 / selected λ\lambdaT=0.4T=0.40.20.1
S-arch (selected λ=0.5\lambda=0.5)17.3 / 23.2 / 23.017.5 / 25.3 / 25.117.5 / 25.0 / 27.0
DiT (selected λ=0.25\lambda=0.25)17.0 / 24.5 / 24.118.0 / 21.7 / 25.417.3 / 10.2 / 26.0

Why λ\lambda is needed in practice. Proposition 1 assumes an exact denoiser, and no trained denoiser is exact. Its logits are inevitably slightly inconsistent with the analytic term of Eq. (10), whose magnitude grows as TT decreases, so the inconsistency is amplified at low temperature. λ<1\lambda<1 damps the amplification by trading it for a known deviation, a slightly sharper bridge qt,Tγq_{t,T}^{\gamma} with γ=β−λ(β−1)\gamma=\beta-\lambda(\beta-1), which vanishes at λ=1\lambda=1. λ\lambda is selected once per model on held-out data, never on the test set, and the held-out sweep already anticipated the DiT result. Moreover, most of the gain comes from the corrected velocity, which is exact for every λ\lambda (next item).

In the revised PDF: role of λ\lambda (§3.1); λ=1\lambda=1 and the held-out sweep (App. C, Tables 2–3).


Q2 (a): Separating TFT, the corrected velocity, and the posterior correction

The corrected velocity carries most of the gain; TFT does not help alone but keeps that gain under strong tempering; the posterior correction adds to it, most at the lowest temperature. The samplers differ only in the three switches shown in the table. At T=0.1T=0.1 (generation seed 1):

SamplerDenoiserVelocityCorrectionS-archDiT
StandardCont.utu_tno17.717.1
StandardTFTutu_tno17.415.8
BCT-PTFTut,Tu_{t,T}no22.723.8
BCT-LTFTut,Tu_{t,T}yes25.926.2
BCT-PCont.ut,Tu_{t,T}no3.917.7

Without TFT (last row), the corrected velocity visits intermediate states the denoiser was not trained on and degrades.

In the revised PDF: all temperatures (App. K, Table 11).


Q2 (b) / W2: Variability across training runs

The gains are robust to training seeds. We re-trained both continuations on S-arch with two more training seeds, keeping λ=0.5\lambda=0.5 without re-selection. For every training seed, BCT-L outperforms standard tempering at every T<1T<1, and the standard deviation of BCT-L across training seeds (0.3–0.8 points) is far below its gain (6–9 points at T≤0.2T\le0.2). We will report training-seed variability for all settings in the camera-ready version.

In the revised PDF: App. L, Table 12.


W2: The vMF flow at T=1T=1

The drop (21.6% vs. 23.6%) does not contradict Proposition 1: at T=1T=1, the two entries use the same sampler and differ only in the checkpoint. Every BCT correction is the identity at T=1T=1 (Proposition 1(ii)); TFT is trained on tempered bridges over a range of temperatures, the released and Cont. checkpoints on the original bridge only. Continual training itself is not the cause (Cont. matches the released checkpoint), and of the six TFT checkpoints in this work, this is the only one whose T=1T=1 accuracy differs from its matched Cont. checkpoint by more than 0.5 points. It is thus a property of this learned denoiser, outside the idealized setting of Proposition 1. Finally, it has no practical cost: BCT reduces to the original sampler at T=1T=1, so the base checkpoint can be used there.

In the revised PDF: App. G, Table 8; Proposition 1 (§3.2).


W2: Scope of the robustness claim

We now state it precisely: at every T<1T<1, BCT-L never falls below standard tempering with the same continual training, in all four settings and, on S-arch, for each of three training seeds. The Limitations now state that Proposition 1 covers the idealized sampler and that robustness to a learned denoiser is an empirical finding, established on two tasks, GSM8K and Sudoku.

In the revised PDF: §4.2; Limitations (§6).


We hope these results address the reviewer's concerns and are happy to clarify further.

Reviewer_f2GvOpenReview ↗

Thank you for the Rebuttal

Thank you for the thorough response and the additional results. They address most of my concerns.

AuthorsOpenReview ↗

Thank you for the follow-up

Thank you for the follow-up.

We are glad that our rebuttal response addresses your concerns: the exact correction, the role of each component, training-seed variability, the vMF case at T=1T=1, and the empirical scope of our claims. Your questions led directly to these experiments and have made the evidence of the paper considerably stronger and clearer.

Thank you again for your careful and constructive engagement through the review.

Reviewer_4n6oOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Tempering of flows for generatuve models specifically applied to language are considered and an inconsistency is addressed ensuring the endpoint and intermediate as well as training distributions are in synch. Advantage of this being a level of insenstitivyt to the degreee of tempering which is illustrated empirically.

Critical strengths

The powered posterior is defined for a flow in a systemic manner than one that is post-hoc

The systemic construction looks to be emirical robust to degrees of tempering speed

Critical weaknesses

There is no insight as to why it would be expected that this construction should be superior to existing schemes

There is no theoretical analysis provided to give some guarantees of e.g. robustness

Questions for authors

Martin Wainright recently released a manuscript with a scheduling scheme - complete with analysis - which looks to be a direct competitor - how do they rate against each other empricially

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4n6o

We thank the reviewer. In short, the revision adds a formal guarantee that explains why the consistent construction is superior and why it is robust to the degree of tempering (Prop.1), and it clarifies that Wainwright's scheduling scheme pursues a different goal from our BCT.


W1 & W2: Why should BCT be superior, and what does it guarantee?

Standard tempering mixes two different processes; BCT does not, and Proposition 1 proves that it then samples the intended distribution at every temperature. The original flow is a mixture of per-token bridges, and averaging their conditional velocities with the posterior moves the mixture exactly. Standard tempering, used by existing flow LMs, weights tokens as if the bridges were tempered but keeps the velocities of the untempered bridges. BCT keeps the weights and the velocities consistent. Proposition 1 makes this precise: with an exact denoiser and exact integration, for every T>0T>0, (i) BCT-E, BCT-P, and BCT-L (λ=1\lambda=1) generate the tempered distribution πT\pi_T exactly; (ii) at T=1T=1, all three reduce to the original sampler; (iii) for the symmetric bridges studied here, standard tempering carries the source along the untempered bridges, so its endpoint is not guaranteed to be πT\pi_T.

Robustness. Since the guarantee holds for every T>0T>0, the correctness of BCT does not depend on the degree of tempering in the idealized setting. In practice, TFT preserves it under strong tempering by training the denoiser on the states that the sampler visits.

Evidence. BCT-P and standard tempering share the weights and the checkpoint and differ only in the velocity; this alone raises the best accuracy by about 6 points (S-arch) and 10 points (DiT). BCT-L never falls below standard tempering at any T<1T<1 in all four settings and for each of three training seeds.

In the revised PDF: Proposition 1 (§3.2); proof (App. H); training seeds (App. L).


Q: Comparison with Wainwright's scheduling scheme

These works pursue a fundamentally different goal from BCT, so they are not competitors; the revised version clarifies this relation in the related work. We believe the reviewer refers to Wainwright (2026), The data geometry of masking diffusion (arXiv:2608.13520), with related manuscripts on Gaussian (arXiv:2607.26285) and product-reference discrete diffusion (arXiv:2608.28949); please let us know if another manuscript was meant.
Wainwright's schedules keep the target distribution fixed and choose where to place a limited number of sampling steps, with a certified KL error. BCT instead defines the target itself, the tempered πT\pi_T, together with the path and velocities consistent with it. In short, schedules decide how to discretize a given target, and BCT decides what is sampled. Since the two do not measure the same thing, a head-to-head comparison would not be informative; at most, a schedule could be applied on top of BCT. These manuscripts also illustrate their schedules on analytic examples rather than trained language models.

In the revised PDF: discussion of certified schedules (§5).


We hope these additions address the reviewer's concerns and are happy to clarify further.

AuthorsOpenReview ↗

Gentle reminder and brief summary of our response

Dear Reviewer 4n6o,

Thank you again for your review. As the discussion phase is ending soon, we summarize below how our response addresses your two concerns and your question.

  1. Why BCT should be superior. Standard tempering tempers the token probabilities but not the dynamics that move the noisy states, so the two no longer describe the same process; BCT tempers both consistently. Correcting the dynamics alone raises the best accuracy by about 6 and 10 points (S-arch, DiT).
  2. A theoretical guarantee. Prop.1 (§3.2; proof in App. H) proves that BCT samples the intended tempered distribution at every temperature, a theoretical basis for the robustness to the degree of tempering that you noted in the review.
  3. Wainwright's schedules. They pursue a fundamentally different goal: given a fixed target distribution, they choose where to place a limited number of sampling steps, whereas BCT defines the tempered target itself and how to sample it consistently. The two therefore do not compete (§5).

Could you let us know whether these points resolve your concerns? If so, we would be grateful if you would consider reflecting this in your rating.

Thank you for your time.

Reviewer_4n6oOpenReview ↗

response

thank you for responding to the comments made will upgrade to score of 3

AuthorsOpenReview ↗

Thank you for deciding to raise your score to 3

Thank you very much for deciding to raise your score to 3.

Your questions on why BCT should be superior and what it guarantees led us to state the guarantee formally as Prop.1 (§3.2, App. H), and the paper is clearer for it.

One request: since the discussion period ends in a few hours, could you kindly update the Rating in your official review as well? The change does not appear there yet.

Again, we are very grateful that you took the time, in a busy period, to read our responses and reply before the discussion closes. Thank you for engaging with our work.

Reviewer_MfZmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper introduces several techniques for implementing temperature scaling in flow-language models. The central observation is that temperature scaling by scaling scores/velocities biases the PDE the model is built on, which distorts the bridge between noisy distributions and, in turn, degrades model performance. To avoid this, the authors identify the quantity that is distorted during standard temperature scaling and correct for this bias in three ways:

  • BCT-L: fine-tuning
  • BCT-P: training-free correction of the path dynamics
  • BCT-E: training-free correction of the endpoint distribution only. Across GSM8K and Sudoku, these methods consistently outperform uncorrected temperature scaling.
Critical strengths

The main strength of the manuscript is that its contribution is mathematically sound and that the result directly addresses a gap in recent work.

S1: Clean Theoretical Addition to Recent Work

Deschenaux & Gulcehre (2026) report that, while they manage to improve the performance of FLMs, a gap between them and block diffusion remains at low temperatures. This manuscript correctly identifies that, because the model class for flow matching differs from, e.g., MLDMs, more sophisticated methods to temper the distribution can be applied. While the authors do not cite the diffusion literature, their contribution appears closely related to the FKC terms derived in Skreta et al. (2025), which are not limited to diffusion models. Investigating connections to the broader literature might help position the work.

Critical weaknesses

The main weakness of the paper is that it is a theoretically narrow incremental result based on very recent work. Since the base architecture of flow language models is very weak compared to other language models, the experimental results lack a relevant baseline, making them hard to interpret and not very convincing.

W1: Theoretical Contribution is Narrow

The paper's theoretical contribution is very narrow. Is an incremental result based on very recent work (Deschenaux & Gulcehre, 2026; Chemseddine et al., 2026). However, even though the theory is presented without citations, the observation that temperature scaling distorts the sampling bridge is not new and has been stated, e.g., in the Feynman-Kac Correctors Paper (Skreta et al., 2025), as well as in others. The fact that this "behaviour" transfers to the model classes studied in this manuscript does not seem particularly surprising. **What is nice, however, ** is that the dynamics of the models considered here seem to allow for analytic correction terms (partly).

Further, it is hard to follow the motivation for proposing BCT-P and BCT-E. To my understanding, BCT-E appears to be the "less corrected" variant and performs differently, but why one would want to half-correct the dynamics is unclear from the manuscript.

W2: Experimental Results Uninformative Without Baselines

The experiments show that temperature scaling improves model performance on language and Sudoku tasks and, to some extent, maintains diversity, as evidenced by increases in pass@k performance.

However, the only baseline given is the base Flow Matching Model, and the (already weak) baselines from the original paper are not shown. At least for GSM8K, it seems like

  • the baseline result at T=1 is about 12-13%, and at lower temperatures the best reported result is below 30% accuracy.
  • the autoregressive baseline achieves >50% in Deschenaux & Gulcehre (2026).
  • a (baseline) block diffusion model by the same authors achieves ~33% accuracy (Deschenaux & Gulcehre, 2026b: BlockGen: Flexible Blockwise Sequence Modelling with Hybrid Samplers), with the same number of steps and using temperature scaling.

Consequently, the gains reported here must be interpreted in the context of other architectures that perform significantly better.

For Sudoku, it would be very helpful to have a random baseline (e.g. the untrained model), as the implementation scaffolding and the Sudoku dataset used can strongly influence the results. In particular, if generation clamps known cells and keeps the full matrix doubly stochastic, this alone can solve some puzzles. Other relevant baselines would be nice too.

W3 (Minor): Presentation makes the manuscript tedious to read.

Many technical concepts are used unnecessarily implicitly and imprecisely. The sampling process is not defined explicitly. For example, the "baseline" temperature scaling is not quite described in equation (3), it is mentioned that logits are scaled, but the actual model parametrisation is not thematised much. The term "bridge" is used a lot, yet the (desired) dynamics of the process and their corresponding continuity equation are not mentioned. Further, the manuscript differentiates between bridge and endpoint tempering, but without the math, the writing is difficult to follow, and the motivation is hard to understand.

I believe the manuscript would benefit greatly from rewriting and from a more explicit use of mathematical concepts.

Also, Table 1 is very hard to read; highlighting the best model version or similar would help a lot. The data seems to be equivalent to Figure 1, so the table could probably be removed entirely.

Questions for authors

Q1: The Motivation of BCT-P vs. BCT-E

What is the motivation for each of the learning-free correction methods? Is one theoretically "more correct"? Why would I prefer one over the other? Is this analogous to the two forms of FKC terms for distribution tempering in Skreta et al. (2026)? If so, then "interpolation" between the path and endpoint tempering seems possible too.

To be clear: I do NOT mean any connection to the SMC resampling in Skreta et al., only the connection to their analytical FKC terms, which are derived from the continuity equation for many operations, including tempering.

Q2: Central motivation

What is the central message/result of the manuscript?

Corrected Temperature Scaling seems to perform better than unscaled Temperature Scaling for this model class. But it seems that the presented flow models are still weaker than comparable alternative architectures. Why should I use an FLM and not, e.g. an MLDM?

4 comments
AuthorsOpenReview ↗

Response to Reviewer MfZm (1/2)

We thank the reviewer for the detailed review. In short: BCT shares the diagnosis of Feynman–Kac correctors (FKC) but differs in target, remedy, and purpose; BCT-P and BCT-E repair the same mismatch from either side, and both are exact; BCT gives flow LMs the temperature scaling that discrete diffusion enjoys, closing about half of the low-temperature gap; the scaffolding alone solves no Sudoku puzzle; and the presentation is revised as suggested. We take "Skreta et al. (2026)" to refer to the ICML 2025 paper.


W1 / S1: Relation to FKC

FKC and BCT share the diagnosis but differ in target, remedy, and purpose.

  • Shared diagnosis. We agree that the diagnosis is known (modifying scores loses control over the marginals; Skreta et al., 2025; Du et al., 2023) and that our discussion of FKC was too brief.
  • Different target. FKC anneals the marginal ptβp_t^{\beta}, over sequences the joint density, which is in general not a mixture of bridges. BCT chooses tempered targets that remain mixtures of bridges and, over sequences, realizes the per-token temperature of autoregressive decoding.
  • Different remedy. Hence FKC needs analytic particle weights, derived from the continuity equation, whereas posterior-weighted velocities transport the targets of BCT exactly (Lemma 1): no weights, a single trajectory. This is the property the reviewer describes as the dynamics "allow[ing] for analytic correction terms".
  • Different purpose. FKC samples a prescribed density with a population of particles; BCT corrects the temperature of the existing single-trajectory sampler at its cost, so its baseline is standard tempering, which every experiment includes.

Beyond this relation, our contribution includes the mechanism—tempering the posterior implicitly tempers the bridge while the velocity stays unchanged (Eqs. (5)–(6))—and explicit tempered bridges, conditional velocities, and exact samplers for two bridge families, plus training on the tempered bridge (TFT); BCT is thus not specific to one model.

In the revised PDF: side-by-side comparison (App. I, Table 9); related work (§5); Lemma 1 (App. H).


Q1 / W1: Motivation of BCT-P and BCT-E

Both repair the same mismatch of standard tempering, from either side, and both are exact; they differ in the states they visit. Training-free variants apply to any released checkpoint at no training cost. The weights of standard tempering assume the tempered bridge while its velocity transports the original one. BCT-P changes the velocity to match the weights and follows the tempered bridge; BCT-E changes the weights to match the velocity and stays on the bridge on which the denoiser was trained. BCT-E is therefore not half-corrected: on its path, the original velocity is the correct one.

  • More correct? With an exact denoiser, both are exact. With a learned one, they differ: on S-FLM, BCT-P gains most at moderate temperatures but collapses under strong tempering, where it visits states sharper than those seen in training, whereas BCT-E stays stable but gains less.
  • Which to prefer? BCT-L if continual training is possible, since training on the tempered bridge keeps the gains under strong tempering; otherwise BCT-P at moderate temperatures.
  • Analogy to the two FKC forms? Not quite: those realize the same path with different splits between velocity and weights, whereas BCT-P and BCT-E are different paths, both without weights.
  • Interpolation? Yes, for a different reason: decoupling the bridge temperature from the endpoint temperature gives an exact, weight-free family between BCT-E and BCT-P. In an exploratory check, intermediate members keep much of the gain of BCT-P while avoiding its collapse, which makes the trade-off explicit; BCT-L removes it by training on the tempered bridge.

In the revised PDF: motivation and Table 1 (§3.2); interpolation (App. M).

AuthorsOpenReview ↗

Response to Reviewer MfZm (2/2)

Q2 / W2: Central message and comparison with other model classes

Central message: in posterior-induced flow LMs, standard temperature scaling is structurally inconsistent—by Bayes' rule, not by model error—and BCT gives this class a tempering that samples the intended distribution. In practice, BCT gives flow LMs the property that makes temperature useful for discrete diffusion: with BCT-L, accuracy keeps improving as the temperature decreases, whereas standard tempering plateaus at 17–18%. At T=0.1T=0.1, the gap to MDLM and Duo shrinks from 15–19 to 6–10 points, i.e., 46–63% of the gap reported by Deschenaux & Gulcehre (2026) closes.

Baselines. The revision adds MDLM and Duo to the main text, with the AR result (53.9%) in the caption. Flow LMs remain behind AR models, and how close a flow comes to discrete diffusion depends on the model: the vMF flow is already more accurate than MDLM and Duo without tempering, and with BCT-L it is competitive with them at their best temperatures (34.5% vs. 33.7% and 36.2%).

Why FLMs. Flow LMs are not uniformly behind discrete diffusion, as the vMF flow above shows. Our claim is not that FLMs should replace MDLMs, but that flow LMs, studied for properties that discrete diffusion lacks (few-step generation, gradient-based guidance, non-factorized updates; Deschenaux & Gulcehre, 2026; Lee et al., 2026), need a consistent temperature; a recent continuous flow already exceeds discrete diffusion in the same TinyGSM setting (Hu et al., 2026). The inconsistency persists even with a perfect denoiser (Proposition 1(iii)), so it remains as these models improve.

On block diffusion. Deschenaux & Gulcehre (2026) report the remaining gap with respect to MDLM, Duo, and AR; block-wise diffusion appears only in their limitations. Being semi-autoregressive, it is a different model class from the full-sequence S-FLM, whose natural comparison is full-sequence discrete diffusion.

In the revised PDF: baselines (§4.6, Figure 5); vMF flow (App. G, Figure 8).


W2: Sudoku baseline

The scaffolding alone solves nothing. We use the Sudoku-Hard generator of S-FLM unchanged, and generation does not clamp clues or impose constraints: the model generates all 81 solution digits, including the clue cells. An essentially untrained model (one update with learning rate 1.2×10−71.2\times10^{-7}) solves 0 of 2,000 puzzles, with no correct cell, at T=1T=1 and T=0.1T=0.1 with three generation seeds.

In the revised PDF: App. F.


W3: Presentation

We revised as suggested:

  • the sampling process (source, ODE, discretization): §2.1;
  • the parametrization from logits to posterior to velocity: around Eq. (4) (Eq. (3) in the submission), §2.2;
  • the continuity equation in the main text (Eq. (1)), satisfied by the tempered bridges and velocities (App. B.3);
  • path vs. endpoint tempering, contrasted in Table 1;
  • Table 1 of the submission moved to the appendix (now Table 10, App. J), keeping Figure 1.

In the revised PDF: §2; Table 1 (§3.2); App. B.3; App. J.


We hope these results address the reviewer's concerns and are happy to clarify further.

Reviewer_MfZmOpenReview ↗

Thank you for the Rebuttal

I thank the authors for their detailed rebuttal and the many improvements to the manuscript. The additional baselines, theoretical clarifications, and revised presentation address most of my original concerns and put the contributions into context.

While I still consider the contribution relatively narrow and remain somewhat unconvinced of its broader practical impact, I acknowledge the technical merit of the proposed corrections and appreciate the improved experimental evidence.

I therefore raise my rating to 3.

AuthorsOpenReview ↗

Thank you for the updated assessment

Thank you very much for raising your rating, and for a thoughtful and constructive review that has made the paper substantially stronger.

We take the remark on practical impact as guidance for the final version, where we will make explicit three properties that matter in practice and that the current results already support: (i) BCT adds no cost at sampling time; (ii) its training-free variants apply directly to existing checkpoints of posterior-induced flow LMs (Sec.4.2); (iii) and since the inconsistency it removes follows from Bayes' rule rather than from model error, its benefit is expected to carry over to stronger flows, as the results of vMF flow already suggest (Sec.4.5).

We are glad that the revision addresses most of your original concerns, and we greatly appreciate your engagement throughout the discussion.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We are grateful to all three reviewers for their careful reading and for comments that led to concrete improvements of the paper. In response, we ran the experiments the reviewers asked for and revised the paper accordingly. The revised PDF is uploaded, with all additions and rewrites shown in blue, and the section, table, and figure numbers below refer to it.

Summary. In posterior-induced flow LMs, standard temperature scaling is structurally inconsistent: it combines the posterior weights of a tempered process with the velocities of the untempered one. BCT removes this inconsistency and thereby gives flow LMs the temperature scaling that discrete diffusion enjoys. Reviewers found the method well motivated (f2Gv), mathematically sound and addressing a gap in recent work (MfZm), systematic and empirically robust to the degree of tempering (4n6o), and the experiments informative (f2Gv).

Main concerns and how the revision addresses them.

  1. Why BCT works and what it guarantees (4n6o, f2Gv). Proposition 1 proves that, with an exact denoiser, BCT samples the intended tempered distribution at every temperature, whereas standard tempering has no such guarantee (§3.2, App. H). The exact correction (λ=1\lambda=1) alone already gives the gains in three of four settings; λ<1\lambda<1 is a safeguard against denoiser error, and most of the gain comes from the corrected velocity, which is exact for every λ\lambda (§3.1, App. C, App. K).
  2. Relation to Feynman–Kac correctors, and the roles of BCT-P and BCT-E (MfZm). The revision deepens the discussion of related work (§5, App. I). FKC and BCT share the diagnosis but not the remedy: BCT chooses tempered targets that remain mixtures of bridges, which are sampled exactly on a single trajectory without particle weights, so its baseline is standard tempering. BCT-P and BCT-E repair the same mismatch from either side, and both are exact (§3.2, Table 1).
  3. Significance and baselines (MfZm). The main text now includes MDLM and Duo, with AR in the caption (§4.6, Figure 5). With BCT-L, accuracy keeps improving as the temperature decreases, closing 46–63% of the low-temperature gap to discrete diffusion; a stronger flow (vMF), already more accurate than discrete diffusion without tempering, is competitive with it at the best temperatures (App. G, Figure 8).
  4. Robustness of the evidence (f2Gv, MfZm). Across three training seeds, BCT-L outperforms standard tempering at every T<1T<1, with a seed variation (0.3–0.8 points) far below the gain (6–9 points) (App. L). An ablation separates the components (App. K), an untrained Sudoku model solves no puzzle (App. F), and the vMF drop at T=1T=1 reflects the checkpoint, not the sampler (App. G).
  5. Presentation (MfZm). §2 now defines the sampling process, the parametrization, and the continuity equation; Table 1 contrasts the samplers; and Table 1 of the submission moved to App. J.

Overall, BCT-L never falls below standard tempering at any T<1T<1 in all four settings (two backbones, Sudoku, and the vMF flow).

Dear Reviewers,

The author-reviewer discussion phase is ending soon. If you have not done so yet, please read the author rebuttals and respond.

Please recall that, as a reviewer, you have agreed to engage in discussion with the authors to clarify and improve the paper. Failure to do so may result in penalties on your own submissions, as per the official ICLR policy. Given the high volume of submissions, your participation is crucial.

Thank you for your time and help.

Best regards,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.