- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The research question is to quantify how large a width is required for the transformer to perform prediction. The authors use predictive-state rank, which is essential for prediction in linear state representation as Hankel rank, discriminated from hidden-state count or belief dimension. The sweep experiments and theoretical analysis demonstrate that predictive-state rank exhibits many important width transitions, even when the hidden-state count or belief dimension remains the same.
Critical strengths
The research direction is interesting to the research community because Transformers are widely used. Its understanding through width perspective in predictive state is worthy of being investigated.
Critical weaknesses
- The results in Table 9 rather indicate that output-side architecture such as smaller width significantly affects the performance, rather than the internal predictive state. The current ablation study, such as Table 8, still does not close this issue.
- What would be fundamentally “width” in a transformer? I mean, there are several dimensions, such as each in-out dimension in QKV and MLP. The controlled experiments on 256 width are performed, but this is still not convincing enough to clearly show the effect of width.
- The width itself is also entangled with several factors, such as gradient dynamics and the optimizer. In other words, a thin model may be affected by several factors, such as the state information in the representation perspective, as the paper claims, different gradient dynamics, different numbers of parameters, and FLOPs.
- The author says dim span{b_t}=K=128, which would not be achieved: under stationary distribution where macro state and clone identity are independent, we have dim span{b_t} \leq g=16.
- Strictly speaking, the second-moment eigenvalue is different from variance. The used C is not covariance; rather, it is an uncentered second moment matrix.
- Line 751-752 (leading eigenvectors): this would be false. After computing the eigenvector, \Sigma^{1/2} should be multiplied.
- Writing should be improved:
- “Equation equation 2”
- “the readout is bias-free, all other linears carry bias.”
- “Belief R^2, state accuracy, and per-mode R^2 are ridge probes” → they are not ridge probes themselves.
- “stated threshold” → ?
- “trivially predictable” → some explanation is required.
- “several known HMMs of known predictive rank”
- Incorrect pages for reference “Value-directed compression of POMDPs”
- Incorrect issues for reference “Computational mechanics: Pattern and prediction, structure and simplicity”
Questions for authors
See the weakness above. Overall, I acknowledge that the authors performed several controlled ablation experiments to address my questions in advance; nevertheless, the current results are still not convincing enough to disentangle the effect of width from compound dynamics. Several measurements also depend on incorrect assumptions.
8 comments
Thank you for the substantial revision and additional experiments. The mathematical corrections, clearer definitions, and new architectural controls have addressed most of my earlier concerns. However, a few important issues remain:
- The experiments still do not clearly distinguish predictive rank from the minimum number of HMM states needed for the same observations.
- The loss decreases gradually around (d=r), and the new scaling experiments do not meet the authors' own predefined criterion.
- The results may still be affected by incomplete training, so it remains unclear whether the observed performance differences are truly caused by limited representation capacity.
Thank you for the careful follow-up, and for acknowledging that the corrections and controls addressed most of your earlier concerns. We appreciate the remaining points and have run additional experiments to address them, as described below. All of these results are included in thelatest version.
1. Predictive rank versus minimal state count. Thank you for highlighting this distinction. We agree that, in our families, predictive rank and minimal state count coincide, and we state the claim at exactly that level in the abstract and introduction. What our experiments do establish is that the width transition reflects the observable process rather than how it is realized: splitting each state into clones, or adding 112 belief directions that never affect predictions, leaves the transition unchanged, while raising the predictive rank moves it.
2. Gradual decrease and scaling. We agree that the transition is broad, and we describe it as such rather than as a threshold. To examine scaling more closely, we ran an additional rank, , and measured how wide a model must be to reach a given accuracy. That width doubles each time the predictive rank doubles: 6.3, 12.0, and 24.0 at for the same excess loss of 0.015 bits, with the same scaling from 0.015 to 0.020 bits. We do not claim an exact collapse of the full loss curve, since that would also require the plateaus to scale. At a fixed data budget, the plateaus are set by data rather than width: fourfold more data lowers the gap by 55–58% but the gap by only 1–4% (App. Table 12).
3. Training versus capacity. This is an important question, and we tested it directly. If narrow models were simply undertrained, giving each width its own best learning rate should close most of the gap. We tuned the learning rate separately below, at, and above rank ( to , two seeds). Narrow models do prefer larger rates ( at versus at ), and tuning helps them most: 25% lower loss at , compared with 9% at and 7% at . Even so, the below-rank model retains 3.5–3.7× the loss of the at-rank model, so tuning closes only about a fifth of the gap. Doubling the training steps and matching parameter counts also leave the gap in place (App. Table 12). We therefore believe that, in this regime, the below-rank deficit reflects limited width rather than optimization.
These results are added to §4.3 and Appendices A.16 and A.17. Thank you again for your thoughtful comments, which have helped us make the paper's claims more precise.
Thank you for the detailed and insightful review. We addressed the architectural questions with more than 100 new runs and corrected the mathematical definitions and references.
W1: output architecture (Tables 8 and 9). We extended Table 9's configuration across residual widths, fixed attention width, and compared linear and MLP readouts with and without final LayerNorm. MLP readouts reduce excess loss by 11–40% relative to linear readouts, while all variants retain a broad transition near r, with plateau onsets at d=18–24 (Fig. 1b, c; App. A.16). At fixed K, the transition moves from d≈19 at r=16 to d≈30 at r=32. These sweeps show that the transition persists across readout choices, extending the exact-state calibration and output-only oracle.
W2: the definition of width. Width means residual dimension d. Attention width also scaled with d in the submission. It is now fixed at 64 (four heads of 16), with MLP width fixed at 256. Allowing attention width to scale with d raises the d=8 gap from 0.062 to 0.105 bits and changes the d=16 gap by about 0.002 bits. The fixed-width controls remove the attention-width confound; a substantial residual-width deficit remains.
W3: optimization, parameters, and FLOPs. We added three controlled experiments (App. Table 12):
The narrow-model deficit persists under these controls, supporting a residual-capacity limitation in this regime. The learning rate was not tuned per width; we note this as a limitation (§7).
W4: belief span under stationarity. You are correct. In the identity-clone realization, every stationary joint law makes clone and macro independent, bounding the belief span by g. We constructed a stationary-latent realization of the same token process with a large belief–rank separation. The original macro chain starts at stationarity, but clone identity is correlated with the initial macro state. The token process is stationary, while the joint latent initialization is not. We now distinguish posterior span 128 from predictive-belief span 120, and Hankel rank r=15 from full-state future-test rank q=16 (§2.1, App. A.19).
The new realization resamples clone identity from a macro-dependent distribution. At η=0.5, the predictive-belief span is 120 (119 centered), the posterior span is 128 (127 centered), and Hankel rank remains 15. Its non-predictive component carries 6–8% of centered variance (App. A.19). Per-token Bayes losses on the existing tokens agree to 10−9 bits, so the trained models and Table 1 prediction losses apply to this realization too.
W5: second moment versus variance. We have recomputed the per-mode analysis using Σ=Cov(bt), including the matched null and sensitivity covariate. We explicitly distinguish this covariance from the uncentered second moment C=E[btbt⊤]. The occupancy-versus-Fisher comparison retains a 2–32% occupancy-truncation penalty over d=1–13 (App. A.4). The second moment is explicitly identified where used for homogeneous-span calculations and earlier results.
W6: Fisher-optimal subspace (lines 751–752). We corrected the missing transformation (App. A.3, A.15). If ui is a leading eigenvector of Σ1/2GΣ1/2, the encoder direction is (Σ+)1/2ui and the reconstruction subspace is span(Σ1/2ui). The code used these encoder directions and optimal linear reconstruction, so the numerical values are unaffected.
W7: writing and references. We corrected the duplicated “Equation,” clarified the submitted architecture's bias terms, and described the quantities as “measured with ridge-regression probes.” The resolved-mode threshold is final R2≥0.1. The text now explains why uniform generators and fixed reset positions need no state tracking, and refers to “HMMs whose predictive rank is known exactly.” Reference details are corrected to Poupart & Boutilier, pp. 1547–1554, and Shalizi & Crutchfield, 104(3–4).
We also corrected the stated ridge penalty to 1.0 and recomputed the g−1 projected-recursion column (now Table 8) with a converged occupancy estimate.
Revisions are marked in blue in the PDF. Submitted Tables 6, 7, 8, 9 are now Tables 8, 9, 10, 15.