A Geometric Obstruction to Robust and Accurate Differentiable Classifiers

We ask when a robust and accurate classifier can be represented as a differentiable function. Reading the gradient of a hypothesis as a *sensitivity field*, we observe that a differentiable classifier's field must be conservative — curl-free and path-independent — while robustness under the manifold hypothesis imposes a separate *strong tangency* condition: invariance to perturbations off a data manifold $\mathcal{X}$, represented as the zero-set of $\varphi$. Given an accurate hypothesis $\psi$, we construct the tangent field $F=P_{\varphi }\nabla \psi$, where $P_{\varphi }$ orthogonally projects onto $\ker (J_{\varphi })$, and show it reproduces $\psi$ exactly on $\mathcal{X}$ via path integration. We then characterize when $F$ remains conservative: testing the antisymmetric part of $J_F$ with arbitrary vectors, it decomposes into bilinear forms whose leading term is governed by the Hessian of $\varphi$, i.e., the curvature of $\mathcal{X}$, and vanishes only in degenerate cases (flat manifolds, or $l=1$ normal-normal directions). For the unit circle, this yields a rigidity result under a specific strong tangency condition — any hypothesis continuous at the origin and strongly tangent to $S^1$ must be constant on it — showing the obstruction might be absolute rather than generic. To test our thesis, we relax conservativeness and represent classifiers as path-dependent functionals evaluated by line integrals of a learned field, using an oracle-path scheme for adversarial training and test. On MNIST, FashionMNIST, and CIFAR-10, adversarially trained path-dependent hypotheses achieve substantially higher robust accuracy than gradient-field baselines at comparable clean accuracy, align more closely with a surrogate data-manifold tangent space, and exhibit statistically significant non-zero circulation — direct evidence that the learned fields are not gradients. Together, the theory and experiments indicate that the accuracy–robustness trade-off is, at least in part, a structural consequence of manifold curvature, motivating path-dependent (multi-valued) architectures as a principled alternative to single-valued differentiable classifiers.