Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training

Differentiating a training run gives a local sensitivity, whereas an outer update makes a finite change to its initialization or training parameters. For hard-ReLU gradient descent (GD), we characterize the response when the perturbation radius is proportional to the GD step size. In this regime, integer changes in activation-crossing indices contribute at leading order. Smooth Euler bias shifts each crossing phase, and earlier rounding decisions move later branch boundaries. For piecewise-$C^2$ dynamics with finitely many separated, same-direction transverse events, we derive the discrete crossing indices and an endpoint expansion with uniform $O({\eta }^2)$ error away from recursive phase boundaries. Contractive affine regions admit an explicit finite remainder that, together with complete branch verification, certifies candidate ordering. A strongly convex scalar example reverses both infinitesimal predictions on $10.55\%$ of 9,801 tested meshes, close to the proved asymptotic frequency of $10.53\%$. Coupled-event experiments isolate downstream rounding feedback; a nonlinear-network study shows how prediction accuracy changes with radius and branch validity. Together with local automatic differentiation and uniform flow consistency, the theory gives three sufficient response regimes. It characterizes finite objective comparisons under stated conditions; it does not guarantee optimizer gains.