- Rating
- 2(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether biased 'random' choices in LLMs reflect an internal task state. Using 208 paired random/deterministic prompts over twelve answer domains, linear probes at the answer position separate the two request types, transfer to held-out domains, implicit paraphrases and naturalistic prompts, and replicate across Gemma-2/3, Llama-3.1 and Qwen2.5/3; selective SAE latents track the same state. On Gemma-2, adding the difference-in-means direction moves deterministic prompts into the random regime but does not raise entropy beyond its natural level, while a late SAE direction chosen by entropy attribution shapes the output distribution. The paper then builds a gated rank-1 reader×writer edit, trains conditional LoRA and rank-4 LoReFT samplers, and compares them with naive prompting and String Seed of Thought on five probabilistic instruction-following targets.
Critical strengths
-
Lexical confounds are handled better than in most probing work. Mid-stack probes transfer from explicit to implicit requests at AUC 1.0 with no false positives on mention prompts, while early layers behave like token detectors. On naturalistic prompts with vocabulary Jaccard 0.77, the leave-scenario-out probe reaches AUC 1.00 against 0.90 for bag-of-words. Appendix D.6 adds an informative negative result: the direction does not separate stochastic sampling from arbitrary free choice (AUC 0.50 on Gemma, 0.57 on Llama), so it encodes an open-choice regime. That is a sharper finding than the title conveys.
-
The causal tests on Gemma-2 come with proper controls. The gated edit raises held-out entropy from 1.48 to 2.20 bits and valid mass from 0.479 to 0.665 while deterministic accuracy goes from 0.96 to 0.95. Without the gate deterministic accuracy falls to 0.15, all 50 random readers and all 50 frequency-matched writers do worse, and a trained rank-1 edit reaches 2.31 bits from the mechanistic initialization against 1.76 from random initialization. The switch-versus-shape dissociation is well supported for this model.
Critical weaknesses
-
The benchmark mixes evaluation interfaces. In Table 2, LoRA and LoReFT are trained for each target and scored by sampling from the valid-answer marginal at the first answer position, while naive prompting and SSoT are zero-shot and scored on free generations that include invalid outputs. Reaching the PRNG interval in 25/25 cells is largely a property of that readout, since the installed marginals already have analytic JS of 0.00–0.24×10⁻³. When the same adapters generate freely (Table 16), TV ranges from 0.04 to 0.39 against 0.05–0.08 for restricted draws, Gemma 3-choice is essentially no better than naive prompting (0.327 vs. 0.333), and on the reasoning model TV after the chain of thought is 0.18–0.54. SSoT is also, by its authors' account, unreliable on non-reasoning models below about 8B, so it is a weak reference here.
-
The link from mechanism to control is thin. The writer is selected by gradient attribution on entropy, so its raising entropy is partly built in, and the paper notes that it mainly increases diffuseness. There is no comparison with a reader-gated temperature increase or logit flattening matched on valid mass, which is the obvious test of whether this particular direction matters. The edit transfers only partly to Llama, the mode direction is not a sufficient switch on Gemma-3 or Qwen3, and projecting it out of the trained LoRA branch removes 2.1% of the adapter's gain. The best-performing controller does not use the mechanism the paper identifies, so mechanism-guided control is shown for one model and as a partial effect.
Questions for authors
-
Under free generation with invalid outputs counted, how do the target-specific LoRA and LoReFT samplers compare with SSoT and with naive prompting that states the target, across all five models?
-
How does the gated edit compare with a reader-gated temperature increase or flattening over the valid answer set, matched on valid mass? If the two are close, what does the specific writer direction add?
-
Is the mode direction necessary for the model's own behavior on natural random requests? Ablating it on the base model cuts valid mass to 0.14 (Appendix D.1), which mixes loss of the random mode with general damage. A targeted ablation that separates the two would strengthen the causal claim.
Follow-up to Reviewer MpuE
Thank you again for your detailed review. With the discussion period ending soon, we would like to know whether our response addresses your concerns.
Following your suggestions, the revision adds a matched end-to-end comparison (Table 3; Appendix E.1) and a comparison of the gated edit with reader-gated temperature and flattening matched on valid mass (Appendix E.7). It also adds a necessity test of the mode direction (Appendix E.6) and an analysis of how the learned LoReFT subspace relates to that direction (Appendix E.8). If these address your concerns, we would be grateful if you would consider updating your score. We are happy to answer any remaining questions.