- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work studies simple (2-layer) decoder-only transformers with RoPE learning a Longest Increasing Subsequence dynamic programming task, using interpretability tools like linear decoding and causal interventions to understand how the model fails on length generalization, with extensions to deeper and different transformer models and the Longest Common Subsequence task. The main findings include that the model learned to encode the DP state for the task in residual stream and this representation is causally linked to task performance, and that limited positional exposure especially hurts routing -- retrieving the right numbers from prior sequence.
Critical strengths
- This work offers interesting insights on failures of length generalization on RoPE transformers on a few representative dynamic programming tasks, including what representations are learned/supports successful inference and what limits OOD performance.
- This work extends and compares findings beyond a single-model/single-task setup, including deeper models/different architecture, as well as different tasks.
- The causal experiments are informative at revealing the use of DP-state representation as well as how much OOD performance can be restored by surrogate positional representations.
Critical weaknesses
- Overall, I think the work would be much stronger if it offered a more comprehensive account of how the model solves or fails at the LIS task. As it stands, the overall evidence seem to only partially account for the mechanism that the model learned or failed. E.g., the surrogate position interpolation intervention only minimally restored OOD performance. The work also only focuses the fixed-length training regime, potentially undermining positional generalization learnable during varying-sequence-length training.
- Some findings don't appear very surprising. For example, it can be unsurprising that the L1 representations from the input tokens can degrade more since it has no output supervision and no incentive to unlearn information carried over from residual connections. It's also unsurprising that the model is weaker at further look-back jumps due to their sparsity in training sequences and the bounded length.
- One thing that makes the overall story unclear is inconsistent framing. E.g., sometimes L0 representations are prescribed as retaining robustly preserving information about the DP state (4.1); but later referenced as severely degrading (4.2).
- Some terms are under-defined, e.g., "geometric requirements".
Questions for authors
- Have you considered training models on varying sequence lengths? I'm curious to hear your intuitions on how that might impact the OOD generalization patterns.
- In 5.2, which sequences had restored accuracy (or which output tokens)? It'd be interesting to know if the position interpolation were helping to restore specifically one kind of mechanism, and what other failure exists from more behavioral failure analysis.
Dear Area Chair and Reviewers,
Following Reviewer PsBo's suggestion, we have uploaded a revised PDF with all additions and edits highlighted in blue. None of the original submission's core measurements were altered; the highlighted sections correspond directly to the new reviewer-requested controls (erasure × oracle in Sec 5.1, variable-length training in Sec 6.2, non-DP sorting in Sec 7.2 / App G, and moving the preliminary patching table to App B). Thank you all for your time and feedback.