- Rating
- 3
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper investigates which computational extension a Transformer learns when multiple candidate mappings agree on every sparse training query. It constructs three reference mappings:a global inverse, a support-masked inverse, and a gated-local decoder. and then distinguishes them using Jacobians, held-out directional responses, dense queries, and interventions on training.
Critical strengths
-
The experimental setting makes computational ambiguity directly testable. The authors first establish that the candidate mappings are equivalent on the supervised sparse query set, then construct probes on which they disagree. This gives a examining about matching an algorithm’s training-distribution behavior
-
The paper provides an informative evidence chain. Changes to auxiliary losses, query support size, and dense-query admixtures affect the learned extension in distinguishable ways. The finding that local derivatives and more distant behavior can change separately is informative
Critical weaknesses
-
Being closest to the gated-local reference does not show that the network implements that mapping. Table 8 reports substantial residuals outside the reference family, but they do not fully characterize the learned computation.
-
The central causal result is partly expected, and the theory does not explain it.
Questions for authors
- When the residual outside the reference family is substantial, which identification claims remain justified?
3 comments
thanks for your reply. i keep the positive score
Thank you to Reviewer 33Yd
Thank you for your reply and for keeping your positive score.
Response to Reviewer 33Yd
Thank you for the positive review and for noting that the derivative at training queries and the behavior farther away can change separately. That separation is central to our claims. The task is also a harder form of in-context learning than in-context regression. Our context has no labels, and we prove that the single-layer linear-attention models studied by in-context regression theory stay wrong on some dictionaries, however long the context (App. O), so the result does not follow from that theory.
W1: closest is not implements. We agree, and our claim is about the learned map, not about how it is implemented. Within the three references, the learned map is closest to gated-local at k = 5, 8, 12, 16 and 20. No network of this architecture can equal either support-dependent reference exactly (App. P), and the learned map is far from the global inverse (Fig. 9). "Closest" has a plain meaning: the network reacts less than either inverse to directions that leave the training subspace, and gated-local does not react at all. The network's reaction is 34% of the global inverse's at k = 5 and 7% at k = 20 (general response). The ranking also predicts new behavior: when we move new queries slightly along a direction where the three references disagree, gated-local predicts the network's response best at k = 5, 8 and 12 (p. 5; Table 5). The Table 8 residual does not change which reference is nearest: it adds the same amount to all three squared distances on every query (Sec. 3.1, p. 5). At k = 5 the residual includes drift (App. J, p. 37): the network's derivative at checkpoints 500 steps apart differs by about two thirds of its distance to gated-local. The change lies mostly in directions that sparse data do not constrain and that the three references do not span, and with the training design fixed, the quantities that set the ranking stay put.
W2: partly expected; theory. Two parts of the result do not follow from the setup. First, removing the support-side losses does not change which maps solve the training task exactly, nor which of them has the smallest Jacobian, yet it moves the network: the nearest reference switches at k = 5, and at k = 8 alpha, the coordinate that runs from gated-local (0) to support-masked (1), rises by 0.08 in all five seeds (App. F.2). So training decides the selection, not the set of solutions. The standard model is much closer to the smallest-Jacobian solution than the global inverse is, but a preference for that solution does not predict when the nearest reference switches (App. I). Second, the far part reacts to very small changes in training, and separately from the near part. Making 0.5% of the training queries dense from the start removes about 82% of the sparse-to-dense gap at k = 5 (Fig. 3). On a trained network, dense queries take effect within hundreds of steps, the effect reverses once they are removed, and the near part keeps its place among the three references (Sec. 4.3). We agree that the direction is intuitive, and the paper's analysis explains it: denser training queries force more directions of the derivative to agree with the global inverse (App. I, remark 3).
Explaining the selection itself is the next question for theory (Outlook, p. 10). The result also does not follow from the closest prior work. There the competing algorithms fit the training data differently, so the training loss can favor one of them (Raventos et al., NeurIPS 2023; Park et al., ICLR 2025). Here all three fit every training query exactly, so the loss cannot favor any, yet the training design moves the network between them (general response, point 4). The setting also makes an account of the selection testable: it has to predict which design change moves the network (p. 10).
Question: which identification claims remain justified? These, now listed in App. L (p. 41): "Proved: the three extensions agree on every training query (Proposition 1), so training behavior cannot identify the extension. Measured at k = 3, 5, 8 and 12: sparse-trained networks extend differently from the global inverse. Their response to directions leaving the training subspace is smaller than both inverses' (Fig. 9), and at k = 5, 8 and 12 their outputs on dense queries are sparse-shaped (Fig. 7). Measured: training design moves the extension. At k = 5 the objective and the query support each switch the nearest reference, at k = 8 they move the family coordinates in the same directions, and at k = 5 and 8 the training-query distribution sets the response to dense queries (Sec. 4)." The discussion runs extend the response comparison and the dense-query result to k = 20 (App. Q). These claims rest on distances, Jacobian norms and outputs on dense queries; none of them assumes that the network's derivative is a combination of the three references.