- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces "Probe Guidance," a method designed to enhance the generation performance of continuous diffusion language models. The approach involves training a lightweight MLP on the intermediate hidden states of a pre-trained main model to predict clean data; the main model remains frozen, and gradients from the probe do not affect the backbone. During inference, the method constructs a guidance direction based on the discrepancy between the main model’s final prediction and the probe’s "weak" prediction, thereby reducing the additional backbone computations typically required by conventional autoguidance. The authors conduct experiments using ELF, FLM, and a 1.7B-parameter CFM, reporting improvements in the Pareto frontier for unconditional generation—measured via GenPPL and unigram entropy—as well as higher average accuracy on multiple-choice questions. Furthermore, the paper proposes a candidate-answer scoring function that incorporates a terminal reconstruction term and investigates the conditions under which autoguidance is effective in language models by utilizing "weak" checkpoints from various stages of the training process.
Critical strengths
The method features a simple structure and clear practical utility. By freezing the main model and training a small, separate "probe," the approach can be integrated into existing checkpoints. Furthermore, it reuses the hidden states already computed by the main model, providing a transparent explanation for the resulting computational savings.
There is relatively direct evidence for the gains achieved on a given base model. Since the main experiments do not require retraining the base model, the comparison between the original model and the version with an added probe remains quite clear. Both ELF-L and FLM-B show significant improvements in GenPPL when controlled for unigram entropy.
The observations regarding the mechanism are insightful and include disclosures of certain unsuccessful outcomes. The finding that "later, 'better' weak checkpoints are not necessarily more suitable for guiding the primary model" is particularly noteworthy. The paper also documents instances such as the insufficiency of cosine similarity as an explanatory metric, unstable behavior in ELF (Eliciting Latent Knowledge), and failures in distillation; these disclosures are valuable for understanding the scope and limitations of the proposed method.
Critical weaknesses
-
The guided probability used for MCQA scoring is not sufficiently defined. For categorical predictions, applying Eq. (4) directly to probability outputs gives (\tilde{p}=\lambda p_s-(\lambda-1)p_w). Although its components sum to one, this extrapolation does not guarantee nonnegative components when (\lambda>1). Such an output can still be used to construct a continuous guidance field, but Eq. (6) treats (\tilde{\pi}_\theta) as a probability distribution inside a cross-entropy loss. The paper does not specify whether clipping and renormalization, projection onto the simplex, logit-space guidance, or another transformation is applied. This leaves an important gap between the guidance definition and the MCQA scorer. I am not claiming that the implementation is necessarily incorrect, but the current description is insufficient to verify how the cross-entropy is computed and whether sampling and scoring use the same guidance formulation.
-
The MCQA evaluation does not fully isolate the scoring changes, and several details needed to assess the comparisons are missing. Table 2 includes an unguided baseline with the modified scorer, so it does provide evidence that probe guidance improves performance under that scorer. However, the terminal reconstruction term and shared noise are introduced together, and no guided result without the reconstruction term is reported. Consequently, the claim in Appendix F that this term is important for obtaining the benefits of guidance is not directly established by the presented ablations. A controlled comparison separating these changes would clarify their individual effects and interactions with guidance.
The evaluation protocol also needs more detail, particularly the CFM probe-training configuration, Monte Carlo scoring procedure, and answer-token masking and normalization. In Figure 7, dropout guidance is shown at (\lambda=2), whereas probe guidance is evaluated at both (\lambda=2) and (3); it is unclear whether the baseline received a comparable hyperparameter search. Uncertainty estimates would also help interpret the smaller accuracy differences.
Questions for authors
-
How is the guided probability in Eq. (6) constructed? Given that the probability-space extrapolation in Eq. (4) can produce negative components, could the authors specify the exact transformation used before computing cross-entropy? Is this clipping followed by renormalization, simplex projection, logit-space guidance, or something else? Please also clarify whether the same guidance formulation is used for generation and MCQA scoring.
-
Could the authors provide the complete MCQA evaluation protocol and clarify the robustness of the comparisons? Please specify the CFM probe architecture, training data and number of training steps; the evaluation splits, prompt format, answer-token masking and length normalization; and the number and sampling scheme of time/noise draws, including how noise is shared across candidate answers. Was dropout guidance independently tuned over a range comparable to that used for probe guidance? Paired confidence intervals or repeated evaluations would help assess the smaller gains. For BoolQ, reporting the majority-class baseline, per-class accuracy, and yes/no prediction proportions would help distinguish improved use of question context from changes in answer-prior calibration.
-
Can probe guidance be extended to larger discrete masked diffusion language models, such as LLaDA-8B?(This question does not require you to provide additional experimental data in your answer) The current experiments focus on continuous flow-based language models. Could the authors discuss what changes would be required to apply the probe approach to a discrete masked diffusion model such as LLaDA-8B, particularly in the probe-training objective and the way guidance enters the sampling process? If this adaptation is feasible, a limited evaluation on a frozen pretrained checkpoint would be informative about the method's broader applicability. If it requires substantial reformulation, a discussion of the relevant limitations would also be useful. I view this as an extension question rather than a required baseline for the current continuous-model setting.
Response to reviewer 4wVS
We thank the reviewer for their detailed reading of our work, and for highlighting the simplicity and utility. Below, we’ve addressed all questions.
On the scoring mechanism: We generally simply clip and renormalize the probabilities when they go negative. We do explore some other methods such as reprojection onto the simplex; they all broadly seem to work and do not materially impact the results.This is different from sampling, where we do not make this change and simply update the noised probability state using the guided prediction. For most of sampling, the input state is not a valid distribution as it is interpolated with gaussian noise. We’ve added further details in Appendix F.
Guidance without the reconstruction term: You are correct to point out that we do not include a guidance result without that term. We find that without that term, guidance has basically no effect. When sampling “on-policy”, we expect that term to be small. Near t=1, the data is basically uncorrupted and the model should roughly learn to pass the input through. For a well trained denoiser, this is basically trivial, so without guidance there is no effect (for finite model capacity reasons, it may be just near zero instead of actually zero for practical models).
MCQA protocol: We have added details to Appendix F providing more details on the MCQA protocol. We did indeed tune dropout guidance and found it was fairly stable within the optimal range.
On extensions to masked diffusion language model: In principle, we could apply similar methods by extrapolating on the logits. One thing we found is that logit extrapolation is extremely stochastic. Probe guidance and vanilla autoguidance, for example, do not help much for GPT2-scale autoregressive models, which similarly operate over a discrete manifold. The probe training approach would not need to change much; the probes would still predict a cross entropy conditioned on the hidden state. We thank the reviewer for acknowledging this as an extension that falls outside of the scope of this contribution and plan to address it in future work.