Scaling Width and Scaling Compute Teach Different Algorithms

Adding parameters and adding optimisation steps are both called scaling, and they push a network toward different algorithms. We show this on two tasks that share no structure, each solvable either by a cheap shortcut or by a general rule, over a $4\times 4$ grid of width and step budget at fixed batch size and $64$ seeds per cell. Compute favours the rule: averaged over widths, the fraction of runs that acquire it climbs monotonically with the budget in every width column of both tasks, from $0.030$ to $0.515$ on one and from $0.001$ to $0.264$ on the other. Width favours the shortcut once past a small optimum, falling $0.686\to 0.520\to 0.257$ between $d=32$ and $d=128$ on the first task and $0.500\to 0.319$ past its peak on the second. So “more scale” is not one direction, and a study that grows a single compound budget averages two effects of opposite sign. The outcome distribution turns out to depend on the rule rather than on the resources: for parity, an all-or-nothing computation, $92\%$ of $9,880$ networks land on one of exactly two solutions, each hitting its predicted accuracy ($1-\eta$ and $1$) to within $0.001$ with nothing fitted, and the smooth averaged curve is then the mixing fraction between them, matched by $1-\eta (1-f)$ to an RMS of $0.0018$, on which no individual model lies. For associative recall, which admits partial credit, the same measurement gives graded outcomes, with $40\%$ of runs in the interior. Discreteness therefore requires a rule that cannot be half-learned; the opposition between width and compute does not.