Identity-Exact Multi-Bias Activations: Breaking Optimization Symmetry Without Changing the Pretrained Function

A nominally three-branch activation can collapse to only two learned knot locations. In the $K=3$ multi-bias parameterization studied here, identity initialization ties its two coefficient-$+1$ branches. Because these branches evaluate the same feature and receive equal gradients, AdamW's normalized update is identical for the two branches up to its $ϵ$ term, preserving the tie throughout optimization. This optimizer-induced branch collapse helps explain why additional branches can match or underperform a same-site $K=1$ control despite the model class's greater expressivity. A multi-bias unit replaces $\sigma (z+b)$ with ${\sum }_{k=1}^K{a}_k\sigma (z+b_k)$ along a shared preactivation. With learned real coefficients, a ReLU multi-bias unit is a one-dimensional spline, represents XOR with $K=2$, and, after thresholding, is Boolean complete on finite hypercubes; an exhaustive exact-arithmetic certificate further shows that some five-input Boolean function requires at least eight knots, $K^{\ast }(5)\ge 8$. The deployed fixed-sign parameterization occupies a degenerate corner of this family: alternating $\pm 1$ coefficients restrict the available slope increments, while tied biases create the optimizer symmetry. Separating the knots breaks this symmetry but, in a controlled audit, perturbs the pretrained activation by up to $53\%$ in relative $L^2$. We resolve this conflict with identity-exact nesting, in which one branch starts with coefficient one and additional branches start with coefficient zero at distinct knots. The resulting network exactly preserves the pretrained function at initialization, while the added branches receive distinct coefficient gradients on the first optimization step. Across four frozen ResNet-50 transfer tasks, nested $K=3$ significantly improves over its matched-site $K=1$ control by $0.55$–$1.63$ accuracy points over three seeds and achieves higher training accuracy in every case. On CIFAR-100, a matched fixed-sign replication is significantly worse than that control ($-1.32$ points), while nested is $0.78$ points better: parameterization decides the effect's sign. Nested is never significantly worse than this control in any tested cell: ViT-B/16 is a four-seed statistical tie, and a 30-epoch sweep peaks at $K=1$. These results show that expressivity and optimizer reachability can diverge even within a single activation family.