Growing in the Right Coordinates: Schur-Gated Function-Preserving Network Expansion

Expanding a trained network without changing its current predictions is useful when additional capacity is needed but restarting training is undesirable. In this setting, however, adding new parameters is not enough: the added directions should complement the inherited model, and they should be parameterized in coordinates that the optimizer can use effectively. Existing growth and conditioning methods typically address these two issues separately. We introduce Sketch-Schur-OST, a function-preserving network expansion method that treats representation and optimization geometry jointly. The key idea is to use a single conditional Schur metric to determine which candidate directions add information beyond the inherited tangent space, how they should be transported relative to the existing parameters, and how the resulting coordinates should be conditioned through a full SPD architectural gate. This produces a unified construction in which selection, redundancy removal, and conditioning are different parts of the same local geometric factorization. We show that the construction preserves the inherited function at insertion, separates support creation from conditioning, and yields an exact factorization of the corresponding damped calibration metric. We further develop a shared-sketch formulation and a row-space dual that make the method practical without materializing the full joint Jacobian–candidate design, while making explicit the conditions under which the sketch guarantees apply. Across controlled growth experiments, the unified construction consistently improves finite-budget optimization relative to matched ungated and sequential alternatives, with gains that persist across coordinate budgets, longer training horizons, adaptive optimization, heterogeneous candidate banks, and larger inherited models. The results also clarify the limits of the approach: aggressive undersketching can fall outside the certified regime, and stronger architectures may remain preferable when retraining from scratch is allowed. Overall, the paper argues that function-preserving growth is best viewed not only as a problem of adding capacity, but as a problem of adding useful directions in the right coordinates.