- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Latent Recirculation to refine a fixed-length workspace for implicit language-model reasoning. A frozen smaller model produces question-conditioned features, which a trainable projection converts into latent slots for a larger reasoner. One additional pass feeds the reasoner's deep latent states into its first-layer keys and values to revisit the same workspace. A training-only CoT branch supplies a detached reference for directional alignment, while answer supervision trains the latent interface and reasoner adapters. Experiments on symbolic, arithmetic, and commonsense benchmarks show the highest five-task average among the compared methods on both main backbones, with gains varying across tasks.
Critical strengths
- The idea is interesting. The fixed workspace and prefill-only feedback provide a concrete way to add latent computation without extending the intermediate sequence or generating a reasoning trace.
- The paper is well-written and easy to follow.
Critical weaknesses
- Eqs. (6)-(7) constrain the angle of the update but do not establish that the latent state moves closer to a useful reference. An arbitrarily small update or an overshooting update can have the same favorable angle. Gradients also pass through the previous state in the realized transition, while the detached reference changes as the shared adapters are updated. Therefore, reduced alignment loss need not reflect progressive refinement of a fixed reasoning state. The paper does not measure whether improved alignment actually reduces the reference-state discrepancy or improves answer quality along the update.
- The no-feedback ablation removes both recirculation and CoT supervision, while every supervised alignment variant retains feedback. This leaves unresolved whether the extra pass is needed to benefit from the reference representation. A single-pass model trained with the same CoT-conditioned reference would isolate that contribution more directly. This distinction matters because feedback without alignment performs worse than no feedback on both backbones, and the full method improves over no feedback by only 2.62 and 1.12 average points.
Questions for authors
- How much of the reference displacement comes from the content of the reasoning trace, rather than its added length and the shifted positions of the latent slots? Would a length-matched shuffled-trace control preserve the alignment benefit?
- The direct-alignment baseline uses cosine similarity between states, whereas transition alignment compares vectors centered at the previous state. Could the gain come from removing a shared residual component through this centering? How do the two losses compare in gradient scale under the same weight of 0.25?