Curvature, Residual Scale, and Signal Propagation in Hyperbolic Neural Networks

How do curvature and residual scale jointly determine signal propagation in hyperbolic neural networks? We study intrinsic input sensitivity for residual layers on hyperbolic space of sectional curvature $-{\kappa }^2$. Along an aligned reference trajectory, we derive an exact Jacobi-field expression for the layer differential and identify a unique transition between contraction and expansion. The transition depends on the dimensionless residual displacement $\kappa hv$ and the covariant derivative of the residual field, rather than on curvature alone. We establish sufficient conditions for these regimes to persist under bounded derivative perturbations and turning of the reference direction. A constrained neural parameterization realizes these conditions, and explicit smoothness estimates extend the reference analysis to contracting neighborhoods and finite-time escape from expanding cones. We further construct a nonlinear correction that preserves the reference trajectory and its entire input-Jacobian product while changing the network away from that trajectory. Numerical experiments verify the differential formulas and sufficient bounds. Controlled learning experiments reveal limited nonlinear-task performance for the original parameterization and substantial improvements after adding the correction, including when the reference Jacobian remains fixed throughout training. These results separate certified reference sensitivity from nonlinear learning behavior and provide a setting in which their relationship can be studied explicitly.