- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper divides the internal states preceding answer generation into three carriers, i.e., prompt, visible text steps, and continuous hidden steps, performing paired activation-patching interventions with respect to changes in an input factor. Given the interactions among these carriers, this work adopts random-order attribution rules from cooperative game theory, demonstrating that when a common attribution rule is held fixed across two reasoning modes, unchanged interaction terms cancel out. Empirical evaluations on a synthetic policy reasoning task reveal that as more reasoning steps are replaced by hidden states, the contribution of text-position states increases, whereas the model's overall accuracy and its ability to apply the policy correctly decline.
Critical strengths
- The paper introduces a cooperative game attribution approach based on random orders and demonstrates the cancellation property of non-drifting interaction terms in paired comparisons via Möbius inversion analysis.
- The experiments show that an increase in internal cache attribution does not correspond to correct policy execution by the model; instead, it is accompanied by an elevated spurious response to inapplicable policies and reduced accuracy, providing negative diagnostic evidence for understanding continuous latent reasoning.
- By incorporating input recomputation comparisons and subset experiments on shared padding tokens, this work rules out spurious attribution artifacts caused merely by output format degradation.
Critical weaknesses
- In the analysis of latent reasoning, recent studies have explored the faithfulness and causal flow of continuous hidden-state reasoning, which shares similarities with this paper. For example, a. CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures. b. Thought Anchors: Which LLM Reasoning Steps Matter; c. Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs; d. LAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning. Furthermore, does this fundamentally act as a form of process reward that assigns values to different rollout components? Please further clarify the core innovations of this paper and its distinctions from prior work.
- All experiments are conducted exclusively on the original GPT-2 model with only 124M parameters. The contemporary motivation for internalizing chains of thought is to compress lengthy context compute overheads in large language models with tens of billions of parameters. Small-scale models lack multi-hop symbolic reasoning capabilities, and their accuracy drops significantly when . Are the phenomena observed on such severely degraded, extremely small models a limitation of the continuous reasoning paradigm itself, or simply a consequence of GPT-2's insufficient capacity to sustain stable reasoning?
- The synthetic rule world used in the experiments contains only two entities, two three-hop membership chains, and three policies. This setup is highly artificial with an extremely constrained state space, failing to represent real-world scenarios. Moreover, combining unpaired key-value pairs from different runs in the self-attention mechanism inevitably introduces out-of-distribution feature perturbations. Although the authors provide an initial check by inspecting the probability mass of legal action labels, an analysis of intermediate and hidden representations is lacking, making it impossible to exclude artifacts induced by the hybrid caches.
- The experiments reveal that identical padding tokens account for a substantial positive attribution change at the cache level. The text attributes this to padding tokens absorbing preceding contextual information; however, in causal interventions, this could just as well arise as a positional encoding artifact introduced by the hard-coded fixed slot width. Comparative ablations with varying slot lengths or dynamic masking without padding should be supplemented.
- When comparing different modes, the number of continuous latent steps and the physical length of the text steps change simultaneously. This means that the total computational step budget is confounded with the text slot allocation, making it difficult to decouple the independent effects of these distinct factors.
Questions for authors
Please see the above comments.