acceptodds
Under review as a conference paper at ICLR 2027

How to Guide Your Language Flow

Abstract

We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics. We apply and benchmark this method on continuous diffusion language models, where probe guidance sets a new state-of-the-art performance on unconditional generation. When applied to a 1.7B diffusion language model, probe guidance consistently improves on multiple choice question answering benchmarks. Using our probes, we study the traditional autoguidance setting where the strong model is a weak checkpoint, and find that the weak model must come from a low-entropy region of training. These findings both provide a practical way to improve diffusion language models and shed light on the actual mechanism behind autoguidance, which is currently poorly understood.

Primary area:
generative models
Keywords:
diffusion, language, guidance, flow matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4wVSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces "Probe Guidance," a method designed to enhance the generation performance of continuous diffusion language models. The approach involves training a lightweight MLP on the intermediate hidden states of a pre-trained main model to predict clean data; the main model remains frozen, and gradients from the probe do not affect the backbone. During inference, the method constructs a guidance direction based on the discrepancy between the main model’s final prediction and the probe’s "weak" prediction, thereby reducing the additional backbone computations typically required by conventional autoguidance. The authors conduct experiments using ELF, FLM, and a 1.7B-parameter CFM, reporting improvements in the Pareto frontier for unconditional generation—measured via GenPPL and unigram entropy—as well as higher average accuracy on multiple-choice questions. Furthermore, the paper proposes a candidate-answer scoring function that incorporates a terminal reconstruction term and investigates the conditions under which autoguidance is effective in language models by utilizing "weak" checkpoints from various stages of the training process.

Critical strengths

The method features a simple structure and clear practical utility. By freezing the main model and training a small, separate "probe," the approach can be integrated into existing checkpoints. Furthermore, it reuses the hidden states already computed by the main model, providing a transparent explanation for the resulting computational savings.

There is relatively direct evidence for the gains achieved on a given base model. Since the main experiments do not require retraining the base model, the comparison between the original model and the version with an added probe remains quite clear. Both ELF-L and FLM-B show significant improvements in GenPPL when controlled for unigram entropy.

The observations regarding the mechanism are insightful and include disclosures of certain unsuccessful outcomes. The finding that "later, 'better' weak checkpoints are not necessarily more suitable for guiding the primary model" is particularly noteworthy. The paper also documents instances such as the insufficiency of cosine similarity as an explanatory metric, unstable behavior in ELF (Eliciting Latent Knowledge), and failures in distillation; these disclosures are valuable for understanding the scope and limitations of the proposed method.

Critical weaknesses
  1. The guided probability used for MCQA scoring is not sufficiently defined. For categorical predictions, applying Eq. (4) directly to probability outputs gives (\tilde{p}=\lambda p_s-(\lambda-1)p_w). Although its components sum to one, this extrapolation does not guarantee nonnegative components when (\lambda>1). Such an output can still be used to construct a continuous guidance field, but Eq. (6) treats (\tilde{\pi}_\theta) as a probability distribution inside a cross-entropy loss. The paper does not specify whether clipping and renormalization, projection onto the simplex, logit-space guidance, or another transformation is applied. This leaves an important gap between the guidance definition and the MCQA scorer. I am not claiming that the implementation is necessarily incorrect, but the current description is insufficient to verify how the cross-entropy is computed and whether sampling and scoring use the same guidance formulation.

  2. The MCQA evaluation does not fully isolate the scoring changes, and several details needed to assess the comparisons are missing. Table 2 includes an unguided baseline with the modified scorer, so it does provide evidence that probe guidance improves performance under that scorer. However, the terminal reconstruction term and shared noise are introduced together, and no guided result without the reconstruction term is reported. Consequently, the claim in Appendix F that this term is important for obtaining the benefits of guidance is not directly established by the presented ablations. A controlled comparison separating these changes would clarify their individual effects and interactions with guidance.

The evaluation protocol also needs more detail, particularly the CFM probe-training configuration, Monte Carlo scoring procedure, and answer-token masking and normalization. In Figure 7, dropout guidance is shown at (\lambda=2), whereas probe guidance is evaluated at both (\lambda=2) and (3); it is unclear whether the baseline received a comparable hyperparameter search. Uncertainty estimates would also help interpret the smaller accuracy differences.

Questions for authors
  1. How is the guided probability in Eq. (6) constructed? Given that the probability-space extrapolation in Eq. (4) can produce negative components, could the authors specify the exact transformation used before computing cross-entropy? Is this clipping followed by renormalization, simplex projection, logit-space guidance, or something else? Please also clarify whether the same guidance formulation is used for generation and MCQA scoring.

  2. Could the authors provide the complete MCQA evaluation protocol and clarify the robustness of the comparisons? Please specify the CFM probe architecture, training data and number of training steps; the evaluation splits, prompt format, answer-token masking and length normalization; and the number and sampling scheme of time/noise draws, including how noise is shared across candidate answers. Was dropout guidance independently tuned over a range comparable to that used for probe guidance? Paired confidence intervals or repeated evaluations would help assess the smaller gains. For BoolQ, reporting the majority-class baseline, per-class accuracy, and yes/no prediction proportions would help distinguish improved use of question context from changes in answer-prior calibration.

  3. Can probe guidance be extended to larger discrete masked diffusion language models, such as LLaDA-8B?(This question does not require you to provide additional experimental data in your answer) The current experiments focus on continuous flow-based language models. Could the authors discuss what changes would be required to apply the probe approach to a discrete masked diffusion model such as LLaDA-8B, particularly in the probe-training objective and the way guidance enters the sampling process? If this adaptation is feasible, a limited evaluation on a frozen pretrained checkpoint would be informative about the method's broader applicability. If it requires substantial reformulation, a discussion of the relevant limitations would also be useful. I view this as an extension question rather than a required baseline for the current continuous-model setting.

1 comment
AuthorsOpenReview ↗

Response to reviewer 4wVS

We thank the reviewer for their detailed reading of our work, and for highlighting the simplicity and utility. Below, we’ve addressed all questions.

On the scoring mechanism: We generally simply clip and renormalize the probabilities when they go negative. We do explore some other methods such as reprojection onto the simplex; they all broadly seem to work and do not materially impact the results.This is different from sampling, where we do not make this change and simply update the noised probability state using the guided prediction. For most of sampling, the input state is not a valid distribution as it is interpolated with gaussian noise. We’ve added further details in Appendix F.

Guidance without the reconstruction term: You are correct to point out that we do not include a guidance result without that term. We find that without that term, guidance has basically no effect. When sampling “on-policy”, we expect that term to be small. Near t=1, the data is basically uncorrupted and the model should roughly learn to pass the input through. For a well trained denoiser, this is basically trivial, so without guidance there is no effect (for finite model capacity reasons, it may be just near zero instead of actually zero for practical models).

MCQA protocol: We have added details to Appendix F providing more details on the MCQA protocol. We did indeed tune dropout guidance and found it was fairly stable within the optimal range.

On extensions to masked diffusion language model: In principle, we could apply similar methods by extrapolating on the logits. One thing we found is that logit extrapolation is extremely stochastic. Probe guidance and vanilla autoguidance, for example, do not help much for GPT2-scale autoregressive models, which similarly operate over a discrete manifold. The probe training approach would not need to change much; the probes would still predict a cross entropy conditioned on the hidden state. We thank the reviewer for acknowledging this as an extension that falls outside of the scope of this contribution and plan to address it in future work.

Reviewer_o2ASOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a probe-based method for continuous diffusion language models, using a small model to guide generation. Experiments show improvements in unconditional generation and average accuracy on multiple-choice questions. The authors also observe that early, low-entropy checkpoints provide more effective autoguidance than later checkpoints in the tested models and propose a simple but effective method for guiding continuous diffusion language models.

Critical strengths
  1. The proposed probe method is simple, and the results demonstrate its effectiveness. It can be applied to other continuous language models at a low computational cost.

  2. The paper is well written and easy to follow.

  3. The work presents ablations of key components and settings, including position, size, FLOPs, and checkpoints. These ablations clarify why the method works and can inform future model design.

Critical weaknesses
  1. The technical contribution beyond Internal Guidance should be clarified. The paper should explain which challenges are specific to continuous diffusion language models and how they motivate distinct design choices.

  2. The paper claims that effective autoguidance requires a weak checkpoint from the low-entropy stage. However, the evidence from ELF and FLM is insufficient to establish this as a general conclusion across different architectures and training settings.

Questions for authors
  1. Can the proposed method be extended to discrete diffusion language models such as MDLM? It may be possible to incorporate the probe in logits space. I hope the authors will conduct experiments or provide analysis to investigate whether and how this would work.

  2. Have you evaluated pretraining continuous diffusion language models from scratch with probes attached? How does this compare with adding probes after pretraining?

4 comments
AuthorsOpenReview ↗

Response to reviewer o2AS

We thank the reviewer for their helpful review and for highlighting the simplicity and effectiveness of our approach! Below, we’ve attempted to clarify and answer the raised issues.

Comparison to internal guidance: We discuss internal guidance (and other, related issues) in Appendix A. Broadly, internal guidance does not stop gradient updates from flowing to the base model This change is very important and has strong downstream implications; in internal guidance, loss weighting needs to be carefully tuned (or be complementary with the main trunk, as in RAE), and investigating different layers requires training new model trunks from scratch. This is impractical in the diffusion language model setting ([1] mostly looks at images) for a variety of reasons, especially as we scale model size and training data. First, as we show in Figures 9a and 13a, dLM training tends to be much less stable than image training, which makes adding hidden supervision challenging. Second, weak guidance can fail catastrophically in dLMs in a way that is much less common in computer vision when the weak model gets too strong. Preventing gradient flowing after the probe places a bottleneck on how strong the input features of the model can get, which we find to be quite important. We expand on these points in Appendix A.

On the autoguidance comparisons: We thank the reviewer for raising this. We evaluated these models because we found it interesting to see such similar behavior on such radically different model setups (data vs latent models, different sizes, etc), and we believe this observation is valuable for the dLM community. In response to your point, we have tempered the language to avoid overextrapolating from this observation.

Extending to discrete diffusion language models: The scope of this submission is focused continuous models as this is the setting in which guidance has been successfully and broadly applied for other data modalities like images or video. It seems reasonable that similar methods to probe guidance could work in the discrete setting. However, the scale of that investigation requires follow up contributions in future work.

Training from scratch: We chose to build on top of existing checkpoints so that we could make clean comparisons to existing work. Earlier in the project we explored training from scratch, and this works fine, despite the fact that the input states to the probes are constantly changing. We saw consistent improvements and roughly the same genPPL-entropy Pareto curves when training from scratch. Ultimately we opted for building on top of existing checkpoints since this is a lot more computationally efficient for practitioners.

[1] Zhou, Xingyu, et al. "Guiding a diffusion transformer with the internal dynamics of itself." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

Could you share the training configurations and quantitative results of your early from scratch experiments, ideally including a comparison with training probes on frozen pretrained checkpoints?

AuthorsOpenReview ↗

Response to reviewer o2AS

Here are the details of the from scratch training. One caveat is we found it difficult to replicate the ELF results identically; the differences were generally a 1-2 genPPL poinnts (this is on the main model trunk, not the probe).

Setup. We trained ELF from scratch on OpenWebText using the standard ELF recipe and configs (Muon, EMA 0.9999, self-conditioning enabled), with weak probes attached during training. Each probe is a 3-layer MLP (hidden size 768, ~14.6M parameters). It reads a detached intermediate hidden state. This means the input feature to the probe is changing every step.

Configurations:

ELF-BELF-MELF-L
Training steps100k100k100k
Probe layer(s)10, 1, 2, 4 (jointly)0, 1, 2, 4, 8, 9, 12 (jointly)
Layer used for guidance129

We trained multiple probes so we could compare them. Forthe direct comparison, "post-hoc" probes use the same architecture, trained for 10k steps on the frozen released ELF checkpoints.

Evaluation follows ELF, with 32-step SDE sampling (γ=1.5), 1000 samples, seed 42. We report similar entropies for comparison.

ModelFrom scratch: CFG onlyFrom scratch: + probe guidancePretrained: CFG onlyPretrained: + post-hoc probe guidance
ELF-B21.5 (5.11)19.1 (5.10)23.2 (5.14)21.7 (5.15)
ELF-M29.9 (5.30)29.2 (5.33)24.0 (5.21)18.2 (5.13)
ELF-L23.2 (5.27)23.0 (5.35)26.5 (5.32)22.4 (5.34)

Gen-PPL (entropy).

ModelEntropyCFG only+ probe guidance
ELF-M (from scratch)5.3637.530.8
ELF-L (from scratch)5.3529.723.0

These demonstrate that cotraining works, even though the probe inputs change. We avoid comparing the trained from scratch base model to the released checkpoints due to the small differences, but probe guidance applied to the co-trained model works well. We will add these results and configurations to the appendix.

I thank the authors for sharing these results. I will maintain my score.

Reviewer_KpdwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes probe guidance for continuous diffusion language models. It trains a small MLP on frozen hidden states from the main model and uses the difference between the main model and the probe to guide generation. This avoids running a separate weak model during inference. Experiments on ELF, FLM, and CFM show improved generation quality and better results. The paper also studies why autoguidance works and finds that weak models from low-entropy training stages are more effective.

Critical strengths
  1. The method is simple and efficient. It uses a small mlp probe instead of a separate weak model, reducing the extra inference cost while improving generation quality.
  2. The paper provides useful ablations on probe design and training, and studies the training dynamics of autoguidance. These results help explain when the method works.
Critical weaknesses

The evaluation is mainly based on generation quality metrics and likelihood-based multiple-choice QA. It would be more convincing to test the method on stronger and more general continuous diffusion language models, such as Sigma-3B[1], and evaluate practical tasks such as math reasoning and code generation.

[1] Large Language Continuous Diffusion Models

Questions for authors

Whether probe guidance can be combined with other guidance methods? Would the two methods provide complementary benefits, or could their guidance signals conflict?

Could you evaluate probe guidance on a stronger model, such as Sigma-3B, using practical benchmarks for math reasoning and code generation? This would help show whether the method improves actual task performance beyond likelihood-based evaluation.

2 comments
AuthorsOpenReview ↗

Response to reviewer Kpdw

Thank you for your thoughtful review and for highlighting the simplicity and efficiency of our method. Below, we’ve addressed your comments and suggestions regarding our metrics and evaluations.

Regarding evaluations on other continuous dLMs, we note that Sigma-3B was first posted on Arxiv on October 1st/2nd, one week after the ICLR deadline, making it impossible to include in this study. At the time of submission, most existing diffusion language models primarily use GenPPL and unigram entropy for evaluation; CFM was the first (to our knowledge) model to go beyond this evaluating on MCQA tasks and also was the largest continuous dLM available at the time (at 1.7B parameters), which is why we include both evaluations in our work.

Regarding the addition of other guidance approaches, we expect that probe guidance could be combined with other guidance methods like classifier free guidance (see, for instance, [1]). However, the current established method to find appropriate guidance signals is via exhaustive search. While we did try this in some settings and saw improvements, it is computationally expensive and it does not help understand the nature of the guidance method we were exploring on the diffusion language model setting. Guidance methods can often conflict in both norm (i.e., one guidance signal is much larger than the other) and direction, and this is quite idiosyncratic to the base model to which guidance is being applied . We’ve added further discussion in Appendix E1.

[1] Karras, Tero, et al. "Guiding a diffusion model with a bad version of itself." Advances in Neural Information Processing Systems 37 (2024): 52996-53021.

Considering the Sigma-3B checkpoint was released close during the rebuttal, I understand that conducting additional experiments may be difficult. But I still be interested in seeing evaluations on stronger models. Overall, the authors' response has partially addressed my concerns, and I am willing to increase my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of all changes

We thank all reviewers for their thoughtful feedback. We have made the following revisions, which are colored in red in the updated document.

  1. We have expanded Appendix A to clarify the distinctions between probe guidance and internal guidance, particularly the importance of gradient isolation in the diffusion language model context.
  2. We expanded Appendix E.1 to discuss combining probe guidance with other guidance methods.
  3. We expanded Appendix F to clarify our MCQA scoring protocol; in particular, how we actually implement guidance on the probability simplex and the details of our probe training.

Thanks again to all reviewers for your efforts!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.