One Marginal Is Enough: What Doubly Stochastic Routing Buys Hyper-Connections, and What It Costs

Hyper-Connections widen the residual pathway into $n$ parallel streams and put a learned $n\times n$ matrix in front of every layer to mix them. Those matrices multiply along depth, so they have to be constrained, and Manifold-Constrained Hyper-Connections (mHC) constrain them to the Birkhoff polytope of doubly stochastic matrices. That is two constraints, one on the row sums and one on the column sums, and we ask what each marginal is worth. Fixing either one alone already bounds the spectral norm of the whole product by $\sqrt{n}$, at any depth, so a single marginal buys the depth-uniform stability the constraint was introduced for. The second only tightens that bound from $\sqrt{n}$ to $1$, and it is not free: the mixer can then neither replicate a source nor aggregate at a destination, and it can only preserve or dissipate the differences among the streams, never enlarge them. We therefore keep one marginal and release the other, giving One-Sided Hyper-Connections (osHC). We instantiate four variants — two that fix the same marginal at every layer, and two that switch marginals partway through the network — and prove that all four are stable at every depth. Across ten routing constructions, two pretraining corpora and three scales, osHC attains the best validation loss in five of the six settings and beats every exactly doubly stochastic construction in all six, and at the largest scale it takes the best and the second-best zero-shot perplexity on each of five held-out corpora, under either pretraining corpus. Trained osHC routers use the released capability at almost every site, while their depth-composed gain stays inside the proved ceiling.