Structured mHC: Objective-Guided Residual Mixing over the Birkhoff Polytope

Manifold-Constrained Hyper-Connections (mHC) use Sinkhorn–Knopp normalization to approximate doubly stochastic residual mixers; exactly feasible mixers in the Birkhoff polytope compose non-expansively across depth. Doubly stochasticity, however, does not determine how information is exchanged: identity, uniform, and permutation mixers satisfy the same constraint while inducing different cross-stream interactions. We introduce structured mHC, which treats the objective used to select a mixer within the Birkhoff polytope as a design variable. We instantiate the framework with a token-dependent linear objective derived from learned mixer scores and compute its mixers with a fixed-depth, differentiable primal–dual method tailored to the row- and column-sum constraints. Under matched protocols and token budgets within each model scale, our method attains lower late-stage training loss than mHC-Sinkhorn in 64M dense and 198M-A64M MoE MiniMind-3 models. In a separate 7B comparison after 15B training tokens, it attains mean accuracy gains of $2.47$, $2.42$, and $0.59$ percentage points over mHC-Sinkhorn across eight tasks under zero-, five-, and ten-shot evaluation.