acceptodds
Under review as a conference paper at ICLR 2027

Scaling Width and Scaling Compute Teach Different Algorithms

Abstract

Adding parameters and adding optimisation steps are both called scaling, and they push a network toward different algorithms. We show this on two tasks that share no structure, each solvable either by a cheap shortcut or by a general rule, over a grid of width and step budget at fixed batch size and seeds per cell. Compute favours the rule: averaged over widths, the fraction of runs that acquire it climbs monotonically with the budget in every width column of both tasks, from to on one and from to on the other. Width favours the shortcut once past a small optimum, falling between and on the first task and past its peak on the second. So “more scale” is not one direction, and a study that grows a single compound budget averages two effects of opposite sign. The outcome distribution turns out to depend on the rule rather than on the resources: for parity, an all-or-nothing computation, of networks land on one of exactly two solutions, each hitting its predicted accuracy ( and ) to within with nothing fitted, and the smooth averaged curve is then the mixing fraction between them, matched by to an RMS of , on which no individual model lies. For associative recall, which admits partial credit, the same measurement gives graded outcomes, with of runs in the interior. Discreteness therefore requires a rule that cannot be half-learned; the opposition between width and compute does not.

Primary area:
interpretability and explainable AI
Keywords:
mechanism selection, shortcut learning, scaling, optimization dynamics, neural networks, transformers, algorithmic generalization, run-to-run variability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o3zhOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the inductive biases of scaling model width and training time on a synthetic task with spurious features. They find that scaling primarily width encourages the use of the spurious feature shortcuts, while scaling training time results in finding the true solution (with lower loss).

Critical strengths

The inductive biases of different types of scaling is a significant and interesting question, and the use of synthetic settings like parity is an established and effective way to gain insight into such questions in deep learning.

Critical weaknesses

The experiments done by the paper are insufficient to support the claims made. They swept over only one parameter (eta) of the one setting (not testing different sized inputs or rule features), used only one architecture and no sweep of learning rate, batch size, depth, or weight decay. Claims about the effects of width and training time are not really useful without at least sweeping learning rates to be able to compare the results at the best learning rate for each width and training time. In many similar settings I have seen choice of learning rate (or small architectural choices, such as the type of layer norms) completely change the results of experiments. Such sweeps wouldn't require an onerous amount of compute for this setting, in my eyes one of the main benefits of a 'toy' synthetic setting such as this one is that orders of magnitude more training runs can be done at little cost of compute.

Additionally, even if the current experiments were better, the paper would still read more like a workshop paper, since it is rather single note, neither connecting to more practical settings than parity nor advancing the state of deep learning theory through further theoretical understanding.

Questions for authors

Clarification: Did you sweep hyperparameters besides width and training time? Potential Improvement: Sweeping over at least lr, doing so well enough to know that finding better lrs wouldn't change results.

Other potential improvements would be showing that the results carry over to more settings, including other synthetic settings but also more applied ones (especially interesting if it can be shown that the results apply to applications such as finetuning LLMs). Alternatively (or even better, additionally) theoretical understanding could help the paper, as long as it actually increases our understanding, such as being useful for making new specific verifiable (and non-trivial) predictions that could then be verified, leading evidence that the theory actually leads to real understanding.

4 comments

Thank you for the specific and actionable suggestions. We ran the learning-rate sweep you asked for at every width and training length, together with batch size, depth, weight decay and a second rule size; the results sharpen the paper's central claim and turn it into a tested prediction.

Every seed is trained under a budget-independent schedule (100 warm-up steps, then constant learning rate) at seven learning rates from 3e-5 to 3e-2, at every width, so that all budgets are read from the same run (Table 2, Appendix E); the one-cycle protocol of the main map is repeated at peak rates 3e-4, 1e-3, 3e-3 and 1e-2 (Appendix E). The grid brackets the optimum: above the best rate of each cell f falls monotonically with the learning rate, below it the rule is still being acquired when the budget ends.

Steps favour the rule everywhere: f is non-decreasing in the budget in every width column at every learning rate on both tasks (88 consecutive-budget comparisons on parity, 64 on recall).

At every fixed rate from 3e-4 to 3e-2, parity at d=128 lies below the best width in every cell where more than 10% of seeds acquire the rule at some width (e.g. 1e-3: 0.86 at d=64 vs 0.73 at d=128), and the same holds under one-cycle at 1e-3 and 1e-2. The optimal width shrinks as the rate grows: d=128 at 1e-4, 64 at 3e-4 and 1e-3, 32 at 3e-3, 16 at 1e-2.

The best learning rate per width and budget falls with width (3e-4 for d ≤ 32, 1e-4 for d ≥ 64 at 5000 steps), and at those rates f is 0.84, 0.86, 0.89 and 0.92 for d = 16–128. This is what carries the width effect at a fixed rate: a wider network at the same learning rate takes a larger effective step, since under standard parametrisation an Adam update moves a hidden unit's input in proportion to its fan-in.

This account predicts that scaling the learning rate as 1/d removes the width effect, and the experiments verify it. With lr = 3e-3·32/d, f at 5000 steps is 0.51/0.72/0.76/0.85 for d = 16/32/64/128, against 0.64 and 0.23 for d = 16 and 128 at the fixed 3e-3; with 1e-3·32/d it is 0.72/0.82/0.87/0.86 (Table 2).

At matched compute, that is at equal training FLOPs, the narrower network trained longer learns the rule at least as often in all 21 comparisons, at each of the seven learning rates and also when each width is trained at its own best learning rate (Section 5.1). This comparison trades width against steps at a fixed budget, and the longer-trained network is never behind.

We tested batch sizes 128 and 512, depth 4, weight decay 0 and 0.1, and a four-bit parity rule (Appendix E). In every variation f never decreases with the budget at any width, and at the final budget d=128 lies below the best width: at 5000 steps, depth 4 gives 0.61 (d=32) vs 0.18 (d=128); weight decay 0.1 gives 0.65 (d=64) vs 0.51; batch 128 and 512 give 0.46 and 0.89 (d=32) vs 0.11 and 0.50 (d=128). Four-bit parity is learned more slowly (after 10⁴ steps 0.14 at d=32, 0.02 at d=128), again with d=128 lowest.

The revised paper goes beyond a single note. It contains a resource map on two unrelated tasks; the step-size account of the width effect with a prediction that the experiments confirm, which is the kind of verifiable prediction you suggested; identification of the learned function on all 16 input classes with interventions and per-network memorisation probes, which establishes two discrete outcomes on parity and three on recall; and a parameter-free mixing law for the averaged accuracy. A practical consequence is that the learned algorithm in a width sweep depends on how the learning rate follows the width, which matters for hyperparameter transfer across model sizes (Section 8). The measurement itself carries over to applied settings wherever the candidate algorithms can be named and made to disagree on held-out inputs, as in the heuristic-versus-rule evaluations of fine-tuned language models discussed in Section 2.

On the hyperparameters swept, the submitted version varied width, step budget and η at learning rate 3e-3, batch size 256 and weight decay 0.01. The revision adds learning rate (7 values), schedule (2), batch size (3), depth (2), weight decay (3) and rule size (2), with 64 seeds per configuration and 61,568 new networks in total.

AI use statement (Ethics flag). Thank you for catching the missing mandatory AI use statement in our initial submission; we have added it on page 10 of the revised PDF.

We hope these results address your concerns.

I appreciate the additional experiments, which strengthen the paper. Unfortunately, the paper is still sloppy, and a bit narrow, so I can only raise the score to weak reject. The result is interesting, but I believe that the paper is still missing either additional understanding or extension to more realistic settings.

As a more minor presentation note, the use of η\eta as a parameter of the setting is a bit awkward as η\eta is normally used to represent learning rate.

Additionally, the added AI use statement does not cover the required areas as mentioned here: https://iclr.cc/Conferences/2027/AIPolicyForAuthors

Thank you for reading the revision and indicating the score increase. We have uploaded a new PDF that responds to each point of your comment.

Notation. The disagreement rate is now written ε throughout the text, tables and figures, so η no longer appears in the paper and cannot be read as a learning rate. Learning rates are always given as numbers, such as 3×10⁻³. The change also unifies the notation with the introduction, which already used ε for the same quantity. Our earlier replies in this thread write η for the same rate.

Presentation. We did a thorough precision pass over the manuscript so that every count, average and step window carries its exact scope: the acquisition fractions in Section 7 (0.22 at d=128 against 0.86 at d=16) state the 10⁴-step window of that run explicitly (at 5000 steps Figure 5a gives 0.16 and 0.70), the 61,568 new networks and 49,024 three-bit parity networks state which grids they comprise (with 830 of the 836 non-exact networks computing partial parity), cell-level bounds specify ε-averaging, the best learning rate notes the tie at d=64, and the median-based remark in Section 7 is replaced by an exact count showing that all 2,588 rule-acquiring seeds at 3×10⁻⁴ and 3×10⁻³ first fell below m = −½.

Understanding. The step-size account of the width effect is now measured directly as well as tested. At the fixed rate 3×10⁻³ the root-mean-square change of a unit's input in one step grows from 2.7×10⁻³ at d=16 to 15.8×10⁻³ at d=128, close to d^0.86, and with the learning rate scaled as 3×10⁻³·32/d it differs by at most a factor 1.2 across widths (Section 5.1, Appendix E). At the fixed rate 3×10⁻³ the wider networks, which take the larger steps, leave the shortcut for the rule less often (Figure 5a), while at 3×10⁻⁴, where the steps are small at every width, the four widths have nearly the same acquisition-time distribution (Figure 5b). Scaling the learning rate as 1/d equalises the step and removes the width effect (Table 2). The Discussion now states separately what is measured, what is explained and tested by a prediction, and what is derived from measured premises (Proposition 1).

Practical settings. The step-size result bears directly on how width sweeps are run in practice. Under standard parametrisation with a fixed learning rate, wider models move toward the shortcut, whereas scaling the learning rate as 1/d, the rule that maximal-update parametrisation prescribes for hidden weights under Adam (Yang et al., 2022), removes the shift while keeping the benefit of more steps. Which hyperparameter-transfer rule is used therefore decides which algorithm the wider models learn, not only their loss. The probe construction that detects this, an evaluation set on which candidate algorithms disagree together with a probe of each network's own training exceptions, applies unchanged wherever the candidate algorithms can be named (Section 8).

AI use statement. Following the ICLR 2027 policy page you linked, we expanded the statement on page 11 so that it covers each required area: the required-disclosure categories (none used, with translation and dataset cleaning not applicable), the recommended-disclosure uses (language polishing and literature organisation), manual verification against the experimental records and original publications, and author responsibility for all content. If this resolves the policy concern, we would also be grateful if you could update the Official Review form when convenient, as it currently still displays the initial rating and ethics flag.

Thank you again for the detailed reviews, which led to the learning-rate sweep and to these measurements.

AuthorsOpenReview ↗

Rating on the Official Review form

In your 17 November comment you noted raising your score to weak reject, while the Official Review form above still shows the initial rating of 1. In case the edit did not save, could you update the rating field before the discussion window closes (Nov 18 AoE)? Thank you again for your time.

Reviewer_Mn1wOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how model width and optimization steps affect whether a neural network learns a shortcut or a generalizable rule. Using two synthetic tasks, parity classification and associative recall, the authors find that increasing optimization steps favors learning the correct rule, while increasing network width beyond a small optimum favors shortcut learning.

The authors also observe that, for parity, most networks converge to one of two distinct behaviors rather than intermediate solutions. They propose a simple first-passage model to describe the probability of acquiring the correct rule within a fixed training budget.

Critical strengths

Interesting empirical finding. The observation that increasing model width and optimization steps can have opposing effects on algorithm selection is interesting and potentially important for understanding neural network scaling.

Controlled experiments. The authors independently vary width, optimization steps, and shortcut strength, allowing them to investigate their effects on rule acquisition.

Statistical evidence. The experiments use 64 independent seeds per configuration, and the observed effects are substantial. The authors also reproduce the qualitative findings on a second synthetic task.

Mechanism-oriented evaluation. The disagreement probes provide a useful way to distinguish shortcut behavior from rule-following behavior, and the memorization control helps exclude an alternative explanation.

Critical weaknesses

Limited experimental scale. All experiments use small two-layer Transformers with widths ranging from 16 to 128. It is unclear whether the findings generalize to substantially larger or deeper networks.

Limited task complexity. Both tasks are highly simplified synthetic problems with deliberately constructed shortcuts. Experiments involving more complex tasks or realistic datasets would strengthen the claims.

Weak theoretical explanation. The first-passage model is largely descriptive. Proposition 1 mostly follows from its assumptions and definitions, and the analysis does not explain why increasing network width favors shortcut learning.

Limited robustness analysis. The paper lacks systematic experiments with different model depths, learning rates, optimization configurations, and compute-matched comparisons.

Incomplete identification of the learned algorithm. The disagreement probe alone cannot guarantee that a network has learned parity. For example, predicting the complement of the marker would also achieve perfect disagreement-probe accuracy. Although ordinary test accuracy partially addresses this issue, exhaustive evaluation of the relevant binary configurations or marker-intervention experiments would strengthen the evidence.

Questions for authors

Q1. Validity and completeness of the disagreement probe

The disagreement probe is designed so that the marker is always incorrect, allowing the authors to distinguish the shortcut predictor from the correct parity predictor.

However, a model that simply predicts the complement of the marker would also achieve perfect accuracy on this probe without learning parity.

Although the authors report ordinary test accuracy that helps rule out this alternative, could they provide a more systematic evaluation of the learned input–output function?

In particular, since the relevant parity problem has only 16 combinations of the three binary features and the marker, could the authors evaluate all of these combinations?

Alternatively, could they test whether the learned prediction remains invariant when the marker is flipped while the content tokens remain unchanged?

These experiments would strengthen the claim that the reported rule-learning networks genuinely rely on the correct content features rather than another shortcut.

Q2. Theoretical explanation of the width effect

The first-passage formulation describes rule acquisition through the probability (P(\tau\leq T)), but does not appear to explain how network width influences the acquisition-time distribution.

Can the authors provide a theoretical or quantitative explanation for why increasing width beyond the observed optimum decreases the probability of learning the generalizable rule?

For example, can they characterize how the distribution of (\tau) depends on width, or establish conditions under which wider networks preferentially acquire the shortcut?

Q3. Robustness to optimization choices and compute normalization

How robust is the observed width-dependent behavior to learning rate, training schedule, optimizer, and other optimization hyperparameters?

Could the authors report results with width-specific hyperparameter tuning to verify that the effect is not caused by unfavorable optimization settings for larger networks?

Additionally, do the qualitative findings persist when comparing different model widths under matched training FLOPs rather than matched optimization steps?

2 comments
AuthorsOpenReview ↗

response to reviewer mn1w

Thank you for the positive assessment of the controlled design, the seed counts and the disagreement probes, and for the concrete suggestions, all of which are now in the paper.

Q1 (whether the probe identifies parity: 16 combinations, marker flip). Both experiments are done for every parity network. In the exhaustive evaluation, each network is evaluated on all 16 classes of (b₁, b₂, b₃, marker), 64 random inputs per class. 98.3% of 49,024 networks have exactly the rule's or exactly the marker's truth table on all 16 classes, and no network implements the complement of the marker, which would score m = 1 on the disagreement probe; the remaining 1.7% follow the rule on a strict subset of the disagreement classes.

In the intervention experiments, networks with the rule's truth table keep their output when the marker is flipped on 99.94% of inputs and change it when one relevant token is moved to the other half of the alphabet on 99.93%; networks with the marker's truth table change their output under the marker flip on 98.6% and keep it under the content intervention on 99.1%; redrawing the five irrelevant tokens leaves the output unchanged on 99.2% (Section 4.1). Rule networks therefore read the three relevant content tokens and ignore the marker.

Q2 (why width favours the shortcut). The new learning-rate sweep answers this quantitatively. At a fixed learning rate a wider network takes a larger effective step: under standard parametrisation an Adam update changes each weight by an amount set by the learning rate, so the change in a hidden unit's input grows with its fan-in d. This predicts that scaling the learning rate as 1/d removes the width effect, and it does: with lr = 3e-3·32/d, f at 5000 steps is 0.51/0.72/0.76/0.85 for d = 16/32/64/128, against 0.64 and 0.23 for d = 16 and 128 at the fixed rate 3e-3; a second base rate gives 0.72/0.82/0.87/0.86 (Table 2). Equivalently, the optimal width shrinks as the learning rate grows: d=128 at 1e-4, 64 at 3e-4 and 1e-3, 32 at 3e-3 and 16 at 1e-2 on parity.

The acquisition-time distributions show where the effect sits: at every width and learning rate the median network picks up the marker within 175 steps, and at the fixed rate 3e-3 only 0.22 of d=128 seeds leave it for the rule within 5000 steps (η = 0.2) against 0.86 at d=16, whereas at 3e-4 the four widths have nearly the same distribution (medians 875–1,062 steps; Section 7, Figure 5).

Proposition 1 is now stated with its two premises, and both are measured (only 11 of 6,801 seeds lose the rule once acquired; 98.3% of parity networks have an exact truth table), so it reduces both resource effects to the measured distribution of τ (Section 7).

Q3 (robustness and compute normalisation). The learning-rate sweep covers seven rates from 3e-5 to 3e-2 for every width and budget under a budget-independent schedule, and four peak rates under one-cycle (Table 2, Appendix E). With the rate chosen per width and budget by held-out cross-entropy, the chosen rate falls with width (3e-4, 1e-4, 1e-4, 3e-5 for d = 16–128 at 5000 steps) and f is 0.84/0.84/0.89/0.86 on parity. The budget-independent schedule reads every budget from one run; f never decreases with the budget at any width and learning rate.

We also tested depth 4, weight decay 0 and 0.1, batch sizes 128 and 512, and four-bit parity (Appendix E). In every variation f never decreases with the budget at any width, and at the final budget d=128 lies below the best width: at 5000 steps, depth 4 gives 0.61 (d=32) vs 0.18 (d=128); weight decay 0.1 gives 0.65 (d=64) vs 0.51; batch 128 and 512 give 0.46 and 0.89 (d=32) vs 0.11 and 0.50 (d=128). Four-bit parity is learned more slowly (after 10⁴ steps 0.14 at d=32, 0.02 at d=128), again with d=128 lowest.

At matched FLOPs, that is at equal training compute, the narrower network trained for more steps learns the rule at least as often in all 21 comparisons across learning rates, and also when each width is trained at its own best learning rate; for example at 3e-4, 1.8×10¹² FLOPs give f = 0.86 (d=32, 5000 steps), 0.72 (d=64, 1,250 steps) and 0.00 (d=128, 325 steps) (Section 5.1, Figure 5c).

On scale and task complexity, the study is designed as a controlled measurement with 64 seeds per configuration, 69,888 networks in all, which requires small models; within that design we now vary depth, rule size (three- and four-bit parity) and the full optimiser grid, and the second task uses an unrelated computation. The probe construction (an evaluation set on which candidate algorithms disagree, plus a probe of each network's own exceptions) applies unchanged to larger models wherever the candidate algorithms can be named (Section 8).

We hope the new evidence addresses your questions.

AuthorsOpenReview ↗

response to reviewer mn1w followup

As the discussion period closes tomorrow, we would like to check whether the revision answers your three questions, and we have uploaded a new PDF that adds one measurement bearing on Q2.

Q1 (identification of the learned function). Every three-bit parity network of the new runs is evaluated on all 16 classes of the three relevant bits and the marker, with marker, content and irrelevant-token interventions: 98.3% of 49,024 networks compute exactly the rule or exactly the marker predictor on all 16 classes, and none computes the complement of the marker (Section 4.1).

Q2 (why width favours the shortcut). At a fixed learning rate a wider network takes a larger step, and the new PDF now measures this step directly. At the fixed rate 3×10⁻³ the root-mean-square change of a unit's input in one step grows from 2.7×10⁻³ at d=16 to 15.8×10⁻³ at d=128, close to d^0.86, and with the learning rate scaled as 3×10⁻³·32/d it differs by at most a factor 1.2 across widths (Section 5.1, Appendix E). Scaling the learning rate as 1/d removes the width effect (Table 2), and the optimal width shrinks as the learning rate grows. One correction to our earlier reply: the fractions of seeds that leave the shortcut for the rule at 3×10⁻³ and ε = 0.2 (written η in our earlier reply), 0.22 at d=128 against 0.86 at d=16, are reached within the 10⁴ steps of that run; within 5000 steps they are 0.16 and 0.70, as Figure 5a shows (Section 7).

Q3 (robustness and compute normalisation). The learning-rate sweep covers seven rates at every width and budget under a budget-independent schedule, together with depth, weight decay, batch size and rule size; at equal training FLOPs the narrower network trained longer learns the rule at least as often in all 21 comparisons, also with each width at its own best learning rate (Section 5.1, Appendix E).

We would be glad to clarify any remaining point.

Reviewer_2UWMOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper explores how width and the number of optimization steps affect the function a simple transformer-based model learns. For this, the authors introduce two synthetic tasks for which there is a general rule to derive the solution and a simpler shortcut that works on most of the examples, controlled by some parameter η\eta. They find that increasing the number of training steps favors the general rule, while increasing the width beyond a certain point favors the shortcut. They further investigate the distribution of the learned behaviors across different parameters, find a strong concentration around two outcomes for the parity task, and propose a first-passage explanation for the observed phenomena.

Critical strengths

The overall question is interesting and distinguishing the impact of width and number of optimization steps is relevant for understanding scaling behavior. The opposing effects observed in the experiments on the two synthetic problems are informative.

The paper tests several different combinations of width/optimization steps and parameters η\eta with 64 independent random seeds per configuration, which provides useful empirical evidence and reduces the influence of random choices of training data and initialization on the outcome. The authors investigate two different tasks and the qualitative agreement between them strengthens the empirical findings of the paper.

Analyzing the distribution of the outcomes is useful. The empirical finding that the outcomes tend to concentrate near two distinct outcomes is an interesting result.

Critical weaknesses

Experimental setup of the associative-recall

In the current implementation of the experiments for the associative recall task, it may happen that the instances constructed for disagreement between "cue" and "rule" do not actually exhibit disagreement.

The generated examples just satisfy that the queried key is not in the first position. The value associated with the first key might still equal the value associated with the queried key. As the alphabet only consists of 88 different values, "cue" produces the correct answer in 18\frac{1}{8} of the cases where disagreement is supposed to happen. This directly influences the subsequent analysis of the performance on task 2 and the conclusions drawn from it. The value for mm for a predictor that only uses the shortcut is therefore 218−1=−0.752 \frac{1}{8} - 1 = -0.75. This interferes with the analysis of ∣m∣>0.95\lvert m \rvert > 0.95 when comparing distributions.

Number of optimization steps influences the learning-rate

Changing the number of optimization steps also changes the learning-rate schedule in the current implementation for the experiments on the influence of number of optimization steps. Consequently, the experiments can't clearly isolate the influence of the number of optimization steps from the influence of the choice of hyperparameters. The authors seem to acknowledge this problem in Appendix C for the acquisition time experiments, but still implement a version where the learning-rate is dependent on the number of optimization steps for the other experiments.

Claim that the parity networks can only learn one of two algorithms

The paper repeatedly claims that a network trained on parity can learn one of exactly two functions.

There is convincing empirical evidence that the networks tend to strongly cluster around the two rules, but it is not established that there are only two possible outcomes of the learning process. The paper itself acknowledges that some percentage of trained networks do not fall into proximity of either rule or cue (∣m∣<0.95\lvert m\rvert < 0.95).

Related to this, the authors claim in lines 302-307 that parity cannot be partially computed. This is not justified, as a network could for example compute the correct parity only on a subset of all possible inputs. Therefore, it is not clear that the observed discreteness of the learned networks does follow immediately from the setup of the parity task.

Section 7 then introduces Proposition 1 under the assumption that a rule cannot be partially acquired. The conclusion seems algebraically immediate once one accepts the assumptions which are not sufficiently established for the parity experiment in the paper.

Presentation

Some parts of the paper are difficult to follow, as important details about the core experiments are either missing or unnecessarily hard to find. Definitions and notation are distributed across different sections. For example, in Section 3 the parity task is introduced, but the definition of "parity of the high content tokens", which is essential for understanding the experiment, is postponed to the appendix without even referencing it. For the associative recall task important details like the size of the vocabulary of keys/values and the methods for sampling are not stated in the paper and can only be recovered by going through the Python code in the supplementary material. The abstract is not sufficiently self-contained and although the main claim of opposing effects is understandable, it is very dense and assumes familiarity with notation, concepts and experimental details that are introduced only later in the body of the paper. The paper also makes use of nonstandard, metaphorical terminology that is neither explained nor formally defined. Examples include:

  • Line 342: "descends the cue direction faster" (What is a cue direction? No such direction is measured in the experiments)
  • Line 343: "each speeds up a different one of two competing clocks" (Only one acquisition time τ\tau is measured and what should speeding up a clock mean in this context?)
  • Line 342: "the cheap solution consolidates before the parity search finishes" (neither consolidation nor parity search is defined.)
  • Line 304: "a network either holds the circuit or not" (holds the circuit is not defined, and the paper does not analyze circuits of the resulting networks, only their behavioral outcomes.)

The authors should clarify the precise meaning of those expressions and distinguish experimentally supported findings from speculative interpretations of results.

Minor weaknesses

There are several numerical inconsistencies and overclaims:

  • Line 137: The existence and uniqueness of the parameter ηc\eta_c are not established.
  • Line 146-148: claims that 0<η≤0.40 < \eta \le 0.4 is the "valid range" for η\eta without giving justification for this claim
  • Line 148: The claim that there is no mass at intermediate values of mm, is inconsistent with Figure 2 (a) and the analysis in the same paragraph.
  • Line 231: The claim that past d=32d=32 increasing the width lowers every line, is inconsistent with the rightmost plot in Figure 3.
  • Table 3: The claim that at every step budget increasing the width first increases and then lowers the fraction with correct associative recall contradicts the data for 625625 steps in the table.
  • Line 236: the number of total networks seems to be miscalculated 4⋅4⋅7⋅64=7168≠74244\cdot4\cdot7\cdot64=7168 \neq 7424
  • Line 341: The claimed monotonic increase with η\eta is in contradiction with Figure 5 (b).
  • Section 8: The claims about scaling are too broad given that only two small synthetic tasks were investigated.
Questions for authors
  • Do the results persist when using the same learning-rate (schedule) for different numbers of optimization steps?
  • In Table 1, why do the seed counts not add up to the same total? Why are there only ≈24\approx 24 seeds reported, when it is claimed that 6464 seeds were used for every experiment?
  • How is the threshold m>12m > \frac{1}{2} chosen? Are the subsequent analysis and its conclusion sensitive to this choice?
  • How is the absence of memorization established in Table 2 for individual networks from aggregated statistics over all networks?
4 comments
AuthorsOpenReview ↗

response to reviewer 2uwm 1/2

(1/2)

Thank you for the careful reading, the precise list of line-level issues and the four questions. Every item is addressed in the revision.

For associative recall, the construction now redraws v₁ whenever the queried key is not first (j ≠ 1), in the training data and in every probe, so the shortcut is wrong on every disagreement input, a pure-shortcut predictor has m = −1 exactly, and its population error is exactly η (Section 3). All 5,120 recall networks of the map were retrained under this construction (Table 3).

We also added a per-position probe (accuracy and rate of returning v₁ for each matched position). It shows that recall outcomes are discrete at three levels: 45.4% of networks use the shortcut (accuracy < 0.1 at positions 2–4), 8.1% implement the rule (> 0.9 at all three), and 46.5% implement an intermediate solution that returns v₁ on only 11% of disagreement inputs and is correct 0.46 of the time, nearly equally at the three positions (mean s.d. across positions 0.03). This matches the 5/12 ≈ 0.42 of a predictor that rejects v₁ and chooses among the other three values (Section 6, Figure 4c).

To separate the step budget from the learning-rate schedule, every seed is now also trained once under a budget-independent schedule (100 warm-up steps, then constant learning rate) and evaluated at every budget, so a network at step T is exactly a network trained for T steps under that schedule. The result is unchanged in direction: at 3e-3, f for d=16 rises from 0.03 (625 steps) to 0.64 (5000) and 0.73 (10⁴); across seven learning rates, f is non-decreasing in the budget in every width column on both tasks (Section 5.1, Table 2, Appendix E). Only 11 of 6,801 seeds that ever exceed m = ½ end below it.

We now measure directly whether parity networks compute "exactly two" functions or partial parity. Each parity network is evaluated on all 16 classes of (b₁, b₂, b₃, marker), 64 inputs per class: 98.3% of 49,024 networks have exactly the rule's or exactly the marker's truth table, no network implements a complement, and the remaining 1.7% agree with both solutions on the eight classes where they coincide and follow the rule on a strict subset of the other eight, i.e. they compute parity on some input classes only. The text states these fractions in place of the earlier wording (Section 4.1).

Proposition 1 is restated with its premises explicit, and both premises are measured: (i) seeds keep the rule once acquired (11 exceptions), (ii) 98.3% of networks compute one of the two functions (Section 7).

On presentation, Section 3 now defines the vocabularies (16 content symbols, 8 keys, 8 values), the high tokens (symbol ≥ 8), the label y = (b₁+b₂+b₃) mod 2, all sampling procedures, the disagreement probe, m, the threshold and ηc\eta_c. The abstract is rewritten to be self-contained. The phrases you list are replaced by measured quantities: the step at which m first falls below −½ (at every width and learning rate the median network acquires the shortcut within 175 steps) and the step τ after which m stays above ½ (Section 7, Figure 5).

Continued in part 2/2 below.

AuthorsOpenReview ↗

response to reviewer 2uwm 2/2

(2/2)

l.137, ηc\eta_c. ηc\eta_c is defined operationally as the first crossing of ½ by the linearly interpolated f(η)f(\eta), reported as > 0.40 when f < ½ on the whole grid (Section 3).

l.146–148, range of η. The marker is positively correlated with the label only for η < ½; at ½ it carries no information about the label, and above ½ its complement is the better shortcut, hence 0 < η ≤ 0.40 (Section 3).

l.148 and Figure 2a. The text gives the counts: 93.9% of 896 networks have |m| > 0.95, and the non-polar ones are concentrated at the largest rates (13 of 64 at η = 0.40; Table 1).

l.231 and Figure 3 (right). At 625 steps f ≤ 0.05 in every cell, so that budget sits at the floor; the decrease from d = 32 to 128 holds at 1250, 2500 and 5000 steps, which is how the caption and text now state it (Section 5).

Table 3. The recall table is recomputed (corrected construction). At 2500 and 5000 steps f peaks at d = 64 (0.278→0.163, 0.466→0.372); at 625 and 1250 steps f ≤ 0.08 everywhere, and the text states this scope (Section 6).

l.236. The map has 4 × 4 × 7 × 64 = 7,168 networks (Section 5).

l.341 and Figure 5b. The sentence is replaced by measured acquisition-time distributions per width and learning rate (Section 7, Figure 5).

Section 8. The discussion is scoped to the tasks, widths, depths, budgets and optimiser settings studied (Section 8).

Q1 (same learning-rate schedule for different budgets). Yes, the results persist; see the budget-independent schedule above. The direction of both effects is the same as under one-cycle, and the learning-rate sweep under this schedule (seven rates, every width) is in Table 2.

Q2 (Table 1 seed totals). The submitted Table 1 came from a separate 24-seed run and listed only the seeds in the two groups, so its rows had different totals. Table 1 now lists all 64 seeds of the d = 64 sweep of Figure 2 at seven rates, with a column for seeds with |m| ≤ 0.95, so every row sums to 64. It also compares each group with the exact accuracy of the marker predictor on that network's own held-out set: the groups match to within 0.003 on average (Table 1).

Q3 (threshold). The threshold ½ lies halfway between chance agreement with the rule (m = 0) and the pure rule (m = 1): m > ½ means following the rule on more than three quarters of disagreement inputs. Appendix D recomputes the map for thresholds 0, 0.25, 0.5, 0.75 and 0.9: on parity no cell changes by more than 0.011, because almost every network has |m| > 0.95; on recall the width ordering is the same at every threshold.

Q4 (memorisation per network). The probe is per network (400 of the network's own training exceptions) and is now reported per network. Under the main protocol none of the 5,280 networks with m < −½ answers more than half of its exceptions (largest 0.46), so they behave as the marker predictor on held-out and training inputs alike. The probe detects memorisation where it occurs: at learning rates of 1e-3 and below, 737 of 9,432 such networks fit their exceptions while following the marker on held-out inputs, against 12 of 14,174 at higher rates (Section 4.3).

We hope these changes resolve your concerns about soundness and presentation.

I appreciate the authors' substantial efforts to address the reviews. The new experiments and fixing methodological issues resolved several of my original concerns.

Nevertheless, I'm still concerned that the paper lacks a sufficiently coherent scientific narrative and does not always clearly distinguish between empirical findings, explanatory hypothesis and established conclusions. The exposition also remains very dense and difficult to follow.

While the rebuttal improved the paper considerably, I'm still not convinced that it warrants acceptance. I therefore raised my score to weak reject.

AuthorsOpenReview ↗

Response to Reviewer 2UWM

Thank you for reading the revision closely and for raising your score. We have uploaded a new PDF that addresses both concerns in your comment.

Findings, explanation and conclusions. The Discussion now opens by sorting the results into three kinds and keeping them apart. Measured directly: in every setting we ran, more optimisation steps never lower the fraction of seeds that learn the rule; at a fixed learning rate, width beyond an optimum lowers it and the optimal width shrinks as the learning rate grows; at equal compute the narrower network trained longer learns the rule at least as often in all 21 comparisons; and 98.3% of parity networks compute exactly the rule or exactly the marker predictor, while recall has a third, intermediate solution. Explained and tested: at a fixed learning rate a wider network takes a larger step in each unit's input, which the new PDF also measures directly (close to d^0.86 at a fixed rate, and equal across widths to within a factor 1.2 under 1/d scaling), and this explanation predicts that scaling the learning rate as 1/d removes the width effect, which Table 2 confirms. Derived: Proposition 1 turns two measured premises, that seeds keep the rule once acquired (all but 11 of 6,801) and that networks compute one of two functions (98.3%), into f = Pr[τ ≤ T] and the mixing law, with no fitted constant; the paper states it as a consequence of these measurements, not as an independent law.

Narrative. The paper follows one line of argument. Steps and width at a fixed learning rate move the choice between rule and shortcut in opposite directions (Section 5). Individual runs end on one of a few discrete functions, so what the resources move is the fraction f of seeds that end on the rule (Sections 4 and 6). The width effect is explained by the step size, now measured directly and tested by the 1/d prediction, and Proposition 1 reduces both resource effects to the measured distribution of acquisition times (Sections 5.1 and 7).

Density. We went through the paper for wording that reads as a general claim where the data have exceptions or a narrower scope. A seed that acquires the rule now almost always keeps it (11 exceptions), the width effect at a fixed learning rate is stated as holding beyond the optimal width, Proposition 1(i) is read as f being the distribution function of τ rather than in figurative terms, and the statement that networks ending on the rule first used the marker is now a count (all 2,588 such seeds at 3×10⁻⁴ and 3×10⁻³). Network counts, averages and thresholds are stated with their exact scope, and the disagreement rate has one symbol throughout.

In the new PDF the disagreement rate η of our earlier replies is written ε. We hope the revised Discussion makes the status of each claim clear.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision

We thank the reviewers for detailed and constructive reviews. They asked for three things above all: a learning-rate sweep at every width and step budget, a separation of the step budget from the learning-rate schedule, and a finer identification of the function each network computes. All three are now in the paper, together with the other requested controls. The revision trains 61,568 new networks (64 seeds per configuration throughout); every number is produced by the released scripts.

Steps favour the rule everywhere. Under a budget-independent schedule (100 warm-up steps, then constant learning rate; every budget read from the same run), the fraction of seeds that learn the rule never decreases with the budget, in any width column at any of the seven learning rates on parity and five on recall (Section 5.1, Table 2, Appendix E).

Width at a fixed learning rate favours the shortcut, and the step size carries this effect. At every fixed learning rate from 3e-4 to 3e-2, parity at d=128 sits below the best width in every cell where more than 10% of seeds acquire the rule, and the optimal width shrinks as the learning rate grows (d=128 at 1e-4, d=64 at 3e-4 and 1e-3, d=32 at 3e-3, d=16 at 1e-2). The best learning rate falls with width, and scaling the learning rate as 1/d, as standard parametrisation predicts for Adam, removes the shortcut shift: with lr = 3e-3·32/d, f at 5000 steps is 0.51/0.72/0.76/0.85 for d = 16/32/64/128, against 0.64 (d=16) and 0.23 (d=128) at the fixed rate 3e-3 (Section 5.1).

At matched compute, that is at equal training FLOPs, the narrower network trained for more steps learns the rule at least as often in all 21 comparisons across the seven learning rates, and also when each width is trained at its own best learning rate (Section 5.1).

To identify the function, every parity network is evaluated on all 16 classes of (three relevant bits, marker bit), with marker, content and irrelevant-token interventions and a per-network memorisation probe: 98.3% of 49,024 networks compute exactly the rule or exactly the marker predictor (Section 4.1).

Associative recall now uses a construction in which the shortcut is wrong on every disagreement input; all 5,120 recall networks were retrained. A per-position probe resolves three discrete outcomes: the shortcut, the rule, and an intermediate solution that rejects the first value and answers correctly 0.46 of the time, close to the 5/12 of choosing among the other three values (Section 6).

The further controls are depth 4, weight decay 0 and 0.1, batch size 128 and 512, a four-bit parity rule and threshold sensitivity, and the paper now reports measured acquisition-time distributions for every width and learning rate (Appendices D–E, Section 7).

On presentation, all task and probe definitions (vocabularies, sampling, the high-token parity, the corrected recall construction, the order parameter and threshold, ηc\eta_c) are in Section 3. The abstract has been rewritten to state the main results and their numbers directly (the OpenReview abstract field carries the same text; the title is unchanged), every figure and table has a stand-alone caption, and the terms the reviewers flagged are replaced by measured quantities. Each numerical statement the reviewers listed has been checked against the data and stated with its exact scope.

The individual responses give the details and the location of each change.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.