- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how answer information is transmitted from continuous latent reasoning states to answer generation via introducing BATS, a causal intervention protocol that crosses latent and generated-prefix cache states. Experiments on CODI–GPT-2 support a latent-to-prefix relay in which latent values repeatedly write answer information into contextual prefix states, which are subsequently read by terminal attention heads. Overall, the results characterize how answer transport depends on the intervention, upstream latent states, and decoding boundary
Critical strengths
- The paper addresses a concrete question about how answer information is transferred from latent states to the generated prefix
- The experiments include several useful controls, particularly the separation between donor-answer transfer and destructive sensitivity. The authors also validate the localized mechanism on 64 new examples with fixed intervention settings.
- The comparison across different latent reasoning models is useful
Critical weaknesses
- confusion about core claim:
Mechanistically, a prefix token's contextual K/V is by construction a function of the full prior computation, so (I suppose that) it's unsurprising that the KV cache of "The answer is" already encodes the answer and is read out downstream.
- Does Information Transport Constitute a Reasoning Mechanism?
A previous study (https://arxiv.org/pdf/2606.12689) found that certain patterns previously interpreted as evidence of latent reasoning also emerge in control models without the corresponding recurrence or training curriculum. So it raises my concern:
Is the observed latent-to-prefix relay specific to latent reasoning, or is it simply a more general information-transfer mechanism commonly found in Transformers?
Questions for authors
While the causal intervention protocol is carefully designed, the broader significance of the identified relay remains unclear.
In particular, the experiments establish that answer information can be transferred through contextual prefix states, but can you consider providing limited evidence of how this mechanism relates to the model's actual reasoning process or downstream reasoning performance?