- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies if the final hidden state of an iterative model is enough to represent task-relevant information. It shows that computation trajectories can retain information lost at convergence, though their practical benefits are model- and task-dependent.
Critical strengths
The authors provide a rigorous theory analysis showing that distinct trajectories can converge to the same endpoint while preserving information. The experimental evaluation is carefully designed. The experiments also cover several model families and report both positive and negative results.
Critical weaknesses
The theory relies on specific contractive dynamics, and its generalizability to practical models remains unclear. Moreover, the experiments do not fully rule out the possibility that stronger nonlinear endpoint decoders could recover the same information. The practical evaluation is relatively weak,the main reliability improvements demonstrated on a few RAFT-family checkpoints.
Questions for authors
see weakness