- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors studies alternative solutions learned by transformers on a cyclic state tracking task. The authors find that the task can be completed with two modes. Without regularization, most seeds are dominated by one mode; with regularization, all seeds have balanced energy across both modes and accelerated learning. The authors further show that similar energy distribution across modes does not necessarily guarantee similar functional contribution to the task output. In the few regularized runs that still converges slowly, the final MLP layer builds the weaker mode from the dominant one. This dependence correctly identifies the slow runs among the new runs, and a similar pattern appears in a version of the task with three modes.
Critical strengths
- The directional dependence experiment is pretty neat. Changing one mode before the last layer and checking the other mode after it in both direction gives direct evidence that one mode is built from the other.
- I like how the authors went from descriptive (energy share) to functional test to interventional study.
- Nice message that a mode can carry energy and be decodable without being computed independently. This is a useful caution for interpretability work that relies only on energy or decoding.
Critical weaknesses
- I find the paper very hard to follow, especially when specific terms or jargons were used but never introduced. Although the results seem interesting, I don't think the paper is ready for acceptance in its current form.
- Insufficient context and literature review in the introduction.
- The paper analyzes converged solutions entirely. There is no learning dynamics analyzed at all.
- I find the analyses restricted to the toy setting of the cyclic state tracking task under a specific form of regularization. The authors didn't discuss why they choose this specific form of regularizer either. Without discussion or experiments on how the results here may generalize to other forms of regularization or other tasks, I find the significance of this work very limited.
Questions for authors
- Why do you choose this specific form of regularization?
- Do any unregularized two-mode runs show one-way dependence across modes?
- For those one-way dependence networks, could there be a single-mode phase before the second one emerges?
- What happens if you change regularization strength or applied it in a middle or early layer instead?
Author Response
Thank you for the careful review and constructive questions. We revised the manuscript and uploaded an updated PDF. We address your concerns below and welcome any follow-up questions.
W1–W2: Presentation and context
We rewrote the title, abstract, and Introduction around the question of whether coexisting codes for the same state are formed independently. Section 2 now introduces the task, gives the state-centroid Fourier decomposition and energy definition, and explains the functional and cross-mode assays at first use. We also revised the Results transitions and expanded Related Work to connect the study to Fourier circuits, state-tracking mechanisms, and the interpretation of interventions.
W3 & Q3: Learning dynamics
We added checkpoint analyses for all 25 fast and five slow regularized C5 discovery runs (Appendix Figure 12). In the slow runs, the source-mode signal develops before the target-mode signal, and the forward removal and phase-transfer responses rise before stable convergence. This is consistent with source-first formation. The archived checkpoints do not establish a sharply bounded single-mode phase or show that this sequence causes slow learning.
Q1: Choice of regularizer
We chose an objective that directly encourages the measured harmonic energies to balance. It changes code strength without directly requiring equal contributions to prediction or independent code formation. This makes the contrast between similar endpoint energies and different functional or dependence patterns testable. We clarified this rationale in Section 3.2 and Appendix J.
Q2: Ordinary-training two-mode runs
We applied the same directional interventions to all ten ordinary-training C5 runs with two-mode endpoints (Appendix Figure 11). Their mean forward-minus-reverse effects are −0.143 for removal and −0.049 for phase transfer, compared with 0.721 and 0.753 in the five slow regularized runs. Thus, these baseline runs do not show the comparably strong positive one-way signature seen in the slow regularized group. We do not infer that their pathways are strictly independent.
W4 & Q4: Sensitivity and scope
At λ=0.25, 0.5, and 1.0, the fast-run counts are 17/30, 25/30, and 28/30, respectively; slow runs remain at every tested strength (Appendix Figure 14). Moving the objective from the endpoint to after block 1 or 4 reduces the number of two-mode endpoints to 8/30 or 12/30, versus 30/30 when applied at the endpoint (Appendix Figure 8). We also added width, depth, and C11 comparisons (Appendix Figures 7, 9, and 10). These extend the tests within the cyclic state-tracking family. Other regularization objectives, tasks, and model families remain untested.
Please let us know if these responses address your concerns.