Convex Polynomial Regularization for Learning Representations with Degree-Ordered Power Growth

Learning effective representations requires inducing structured inductive biases over function spaces, where polynomial degree serves as a natural hierarchy of structural complexity. While linear degree weighting enforces this ordering, standard degree-dependent power penalties inadvertently invert capacity control near zero. This study resolve this by introducing two shape-corrected regularizers that preserve global degree monotonicity including a linear-power penalty (LiP) and a shifted-power penalty (ShiP). These formulations directly constrain the learned coordinate representations within coercive, strictly convex optimization landscapes, guaranteeing unique global solutions and admitting exact coordinate-wise minimization. Across 09 regression benchmarks, regularizing representations via ShiP achieves competitive predictive performance that rivals non-parametric Gaussian process baselines. By bridging degree-ordered complexity constraints with non-linear power penalties, this framework ensures that expressive feature representations remain well-conditioned, interpretable, and computationally tractable without distorting the underlying capacity hierarchy.