Which Task Differences Matter? The Geometry of Optimal Shared Iteration

A solver that reuses one learned coefficient schedule across many tasks can underperform because of insufficient training, a restrictive update rule, or missing task information, but its loss cannot identify the cause. For positive-definite quadratics, we separate these sources of excess risk and characterize which task differences change the optimal program, the residual polynomial the schedule implements. Task laws with different spectra and oracle risks can require the same program. Given the population risk of every depth-$T$ program (equivalently, moments through degree $2T$), moment vectors sharing an optimal depth-$K$ program form affine fibers of dimension $\max \{K-2T,0\}$ through each interior point. Across fibers, the leading oracle-relative cost of sharing a program is the squared radius of the smallest ball enclosing the linearized optimal programs in the local risk metric. Under this criterion, the minimax program is optimal for a least-favorable mixture of laws; we prove a sharp worst-case bound, attained by smooth positive densities, on how many laws this mixture needs. A momentum recurrence implements this program while preserving the optimal first $T$ steps shared by all laws in the family.