From Tangent Geometry to Finite-Step Curvature in Safe Policy Optimization

Constrained trust-region methods choose updates using first-order tangent models, but evaluate finite policy steps with nonlinear reward and cost surrogates. We characterize exactly what the tangent model determines and how curvature first alters that decision at finite step sizes. With $m$ linearized constraints, the Fisher trust region's complete reward–constraint score image is preserved by ${\mathcal{S}}_m=\mathrm{range}(H^{-1}[g,b_1,\dots ,b_m])$, the unique smallest linear subspace with this property. Reward–Constraint Subspace (RCS) uses this tangent-exact domain to reoptimize projected reward and constraint curvature, then applies empirical-KL correction and nonlinear same-batch checks before committing an update. We also quantify the approximation from restricting updates to ${\mathcal{S}}_m$: along a regular small-radius branch, the full-space and restricted solutions share the same $O(\sqrt{\delta })$ tangent term and first differ by an $O(\delta )$ Fisher-orthogonal displacement governed by cross-domain active-Lagrangian curvature. The resulting reward-surrogate gap is $O({\delta }^{3/2})$. Under weaker Fisher-relative remainder bounds, projection provides an $O(\delta )$ finite-surrogate comparator for arbitrary $m$. On Safety-Gymnasium, matched curvature diagnostics show improved finite-step model fidelity and update reorientation, while an independent nonlinear full-space reference recovers the predicted leakage magnitude, scaling, and direction.