acceptodds
Under review as a conference paper at ICLR 2027

A Geometric Obstruction to Robust and Accurate Differentiable Classifiers

Abstract

We ask when a robust and accurate classifier can be represented as a differentiable function. Reading the gradient of a hypothesis as a *sensitivity field*, we observe that a differentiable classifier's field must be conservative — curl-free and path-independent — while robustness under the manifold hypothesis imposes a separate *strong tangency* condition: invariance to perturbations off a data manifold , represented as the zero-set of . Given an accurate hypothesis , we construct the tangent field , where orthogonally projects onto , and show it reproduces exactly on via path integration. We then characterize when remains conservative: testing the antisymmetric part of with arbitrary vectors, it decomposes into bilinear forms whose leading term is governed by the Hessian of , i.e., the curvature of , and vanishes only in degenerate cases (flat manifolds, or normal-normal directions). For the unit circle, this yields a rigidity result under a specific strong tangency condition — any hypothesis continuous at the origin and strongly tangent to must be constant on it — showing the obstruction might be absolute rather than generic. To test our thesis, we relax conservativeness and represent classifiers as path-dependent functionals evaluated by line integrals of a learned field, using an oracle-path scheme for adversarial training and test. On MNIST, FashionMNIST, and CIFAR-10, adversarially trained path-dependent hypotheses achieve substantially higher robust accuracy than gradient-field baselines at comparable clean accuracy, align more closely with a surrogate data-manifold tangent space, and exhibit statistically significant non-zero circulation — direct evidence that the learned fields are not gradients. Together, the theory and experiments indicate that the accuracy–robustness trade-off is, at least in part, a structural consequence of manifold curvature, motivating path-dependent (multi-valued) architectures as a principled alternative to single-valued differentiable classifiers.

Primary area:
learning theory
Keywords:
robustness, adversarial, representation, obstruction, trade-off, accuracy

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9KuxOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper views the problem of learning an accurate and robust classifier via the lens of sensitivity field (vector field calculus).

Assuming the data is part of a manifold, the robustness problem can be translated as the change on the classifier to be agnostic to outside the data manifold and only parallel to it, i.e. gradients of the classifier (vector field of interest) should be essentially parallel to the data manifold and insensitive to change in other directions. The authour formalizes this property as the tangency condition.

In general the classifier can be defined as the path integral of this vector field of interest. Moreover, standard classification training enforces this vector field to be naturally a gradient of the classifier leading to a conservative field.

The key insight of the paper is that, more often than not, the tangency condition to ensure robustness can't be simultaneously satisfied by a conservative field our standard gradient vector field induced by conventional classification training. The authors refer this to the geometric obstruction.

The paper provides theorems, and examples to exhibit this perspective. Additionally, they subscribe to a general vector field path integral definition of learning the classifier, and show that its implementation (by learning the non-conservative vector field directly) indeed improves robustness empirically.

Critical strengths
  1. The biggest strength of this paper is the novel perspective of viewing a robust classifier problem by the lens of vector field calculus, and deriving structuaral limitations of standard classification training.
  2. The paper also provides empirical evidence on image classification tasks : a) that indeed the learned robust vector fields directly are not conservative b) alignment of gradient field induced by standard training is worse than alignment of the learned vector fields, bolstering the theoretical argument.
Critical weaknesses

The weaknesses are outweighed by the strengths of this paper, however I would like the authors to address the following questions:

  1. To address the incompatibility of the tangency and conservative field condition, we need to utilize the path integral definition of the classifier (hypothesis). However, as a practical approach this would be computationally infeasible. As the authors themselves have pointed out even for CIFAR 10 the computations can scale up quickly. Moreover, choosing an optimal path for the integrals is also a choice which needs further investigation.

  2. The main issue arises from the use of single value loss functions, which lead to gradient vector fields that are conservative, an alternative way to mitigate that condition could be, by using a multi objective loss? It would be great, if authors could add some discussion and relations to existing multi objective classification loss on how they can by pass this conservative constraint.

Questions for authors

Please refer to the weakness section.

3 comments

To address the incompatibility of the tangency...

This is an important and broad question, and we cannot address it fully within a single paper. What we can say at this point is that, regarding time complexity, a less accurate approximation of the path integral, such as a single-step trapezoidal rule, can be effective.

Regarding memory complexity, a promising direction is to use a Helmholtz decomposition to represent the sensitivity field as a sum of conservative, divergence-free, and harmonic components. The exact form of this decomposition depends on the domain and the boundary conditions. Nevertheless, it could provide a useful way to separate the components of the field and to investigate which of them are necessary for the representation.

Another possible direction is a method that resembles purification, as described in the general response. Such a method could avoid explicitly computing the full path integral in the manner used in our experiments.

The computational complexity and the choice of paths remain open questions. We do not claim that the current implementation is a practical solution to these problems.

The main issue arises from the use of s...

This could be a promising direction, although we have not approached the issue from this perspective.

According to our framework, changing the loss function alone is not necessarily enough to bypass the obstruction, because the obstruction originates in the hypothesis rather than in the loss: a differentiable scalar hypothesis imposes a conservativeness constraint on its sensitivity field. It might nevertheless be possible to derive a Pareto front for the robustness and accuracy objectives based on the presented framework. For example, accuracy may require the sensitivity field to contain components that are incompatible with the desired tangency condition, while improving robustness may require diminishing those components. This would lead to a trade-off between accuracy and robustness.

However, this idea is still too preliminary to be added to the paper, and the mathematical machinery required to investigate it properly would likely deserve a separate paper.

I thank the authors for the response. I urge the authors to have a brief discussion in the limitations and future works pertaining to these questions.

Thank you again for your time and for your encouraging words and review. We have added the discussion to Section 6 of the paper, which will be available for your review as soon as the second revision of the paper is ready.

Reviewer_gQYqOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks whether a robust and accurate classifier can be represented as a differentiable function. It views the gradient of a hypothesis as a sensitivity field, which must be conservative. Under the manifold hypothesis, robustness is formulated as a strong tangency condition: the field should be invariant to off-manifold perturbations. Projecting the gradient of an accurate hypothesis onto the tangent space reproduces it on the manifold, but the resulting field is generally not conservative, with the obstruction governed by the manifold’s curvature. For the unit circle, the authors show that any continuous strongly tangent hypothesis must be constant. They then propose path-dependent (multi-valued) hypotheses, defined as line integrals of a learned field, and report on MNIST, FashionMNIST, and CIFAR-10 that these achieve higher robust accuracy than gradient-field baselines, align better with a surrogate tangent space, and have non-zero circulation.

Critical strengths
  1. The theory is extremely interesting. It addresses a fundamental question: whether a model can be both robust and accurate at all.
  2. The definitions are natural.
Critical weaknesses
  1. Missing related work. The paper does not discuss how its point of view relates to existing theory in the field. For example, there are theoretical works on adversarial examples in random networks and on data with an implicit low-dimensional structure.
  2. Limited experiments. The experiments mostly use small datasets that are nearly linearly separable (MNIST, FashionMNIST). This may be problematic for this framework, since it reduces the problem to the well-studied margin-maximization problem for linear networks.
  3. Weak approximation of the data manifold. The local tangent space is approximated with nearest neighbors and PCA, and no other local approximation methods are explored.
  4. The choice of x₀ is not discussed enough. The result that the mean x₀ performs worse than x₀ = 0 is interesting, but it gets very little discussion.
  5. High complexity. The proposed method is computationally very expensive.
  6. No practical implication. The idea should be developed further into a practical implication, even a partial one, that supports the point of view and connects it to current research.
Questions for authors
  1. How does this framework relate to existing theoretical results on adversarial examples, for example, in random networks or for data on low-dimensional subspaces?
  2. Did the authors consider or experiment with other methods for approximating the local data manifold?
  3. Can the authors explain why the mean x₀ performs worse than x₀ = 0, or provide some intuitive reasoning?
4 comments
AuthorsOpenReview ↗

Response (1/2)

Before turning to the questions, we would like to clarify two points concerning the weaknesses listed in the review.

Maximum-margin classifiers: Maximum-margin classification theory is not at odds with our framework. In fact, it is one of its main sources of inspiration. The paper points to this connection in lines 183--184, where it refers to the boundary-tilting perspective.

Figure 1 illustrates the relevant geometry. The data manifold is flat, and the linear decision boundary achieves the maximum margin with respect to the manifold by intersecting it orthogonally. Since the sensitivity field of a linear classifier is normal to its decision boundary, it is parallel to the data manifold. This is why the sensitivity field is tangent to X\mathcal{X} in the geometry shown in the figure.

In contrast, tilting the decision boundary towards the manifold changes this geometric relationship: the sensitivity field is then no longer tangent to X\mathcal{X}. This is precisely the boundary-tilting perspective, expressed in terms of sensitivity fields.

No practical implication: Some of the practical implications of the paper are the following:

  • Under the assumptions of the paper, a practitioner using a single-valued hypothesis faces a trade-off between robustness, accuracy, and differentiability. In particular, to retain both robustness and accuracy, differentiability may have to be sacrificed. This has significant consequences for the choice of hypothesis space and learning algorithm.
  • A practitioner who wants to retain all three properties can instead model the hypothesis as a multi-valued function. The practitioner does not have to follow our construction and might, for example, use a purification method. However, path dependence is essential to this approach.

How does this framework relate to e...

Trade-off results: Tsipras et al. (2019) construct a data distribution on which no classifier is both robust and accurate. This does not explain how humans classify robustly, and Yang et al. (2020) argue that it does not apply to datasets such as MNIST. Haldar et al. (2024) instead attribute vulnerability to on-manifold inseparability combined with slow off-manifold convergence of first-order optimizers, which is an optimization account. Ours is a representational one: the obstruction we identify lies in the hypothesis space and does not depend on the optimizer.

Random networks: Bubeck et al. (2021), Bartlett et al. (2021) and Montanari & Wu (2023) show that, with high probability, a network with random weights has an adversarial example close to any given input xx along the gradient direction. These results describe a typical hypothesis and make no reference to a target. Our framework addresses the complementary question: the optimal sensitivity field F∗F^* need not be conservative, whereas the gradient of a differentiable scalar hypothesis must be. A Helmholtz decomposition,

F∗(x)=∇Ψ(x)+R(x)+H(x),F^*(x)=\nabla\Psi(x)+R(x)+H(x),

with RR divergence-free and HH harmonic, gives an intuition for how the two views may meet. A scalar hypothesis can reproduce only the part of F∗F^* that is a gradient, so an accurate hypothesis must approximate F∗F^* within the conservative class, which may introduce additional components into the learned sensitivity field. These components may be related to the vulnerability observed in random networks, and the harmonic component to the near-linear behavior that Bartlett et al. (2021) identify as its cause. We offer this as an intuition only; a rigorous treatment belongs in a separate paper.

Low-dimensional data: The paper summarizes the consequences in lines 272-279. If the geometric obstruction reflects a topological defect, no continuous change of coordinates can remove it, since homeomorphisms preserve topology (no homeomorphism maps a circle onto an interval). A nonlinear representation only relocates the problem: polar coordinates flatten a circle, but require a cut in the angular coordinate and are singular at the origin. Removing the obstruction globally therefore requires a representation that is discontinuous or singular, which continuous models such as ANNs or polynomials may struggle to learn.

AuthorsOpenReview ↗

Response (2/2)

Did the authors consider...

Yes. We have also performed the experiment with the tangent space approximated using image invariants, following the idea of Tangent Prop (Simard et al., 1991). This experiment produces similar results, and we will add it to the appendix.

This method can approximate the tangent space accurately when a substantial bank of transformations is available. However, the universally applicable transformations (x-translation, y-translation, rotation, scaling, axis-aligned shear, and diagonal shear) form a relatively small set. They therefore fix the dimension of the estimated tangent space to six for all datasets, irrespective of the individual sample.

The PCA method, on the other hand, allows the dimension of the estimated tangent space to vary from sample to sample, although its accuracy is limited by the availability of data. The two methods thus have different advantages: image invariants provide a more structured approximation when suitable transformations are available, whereas PCA allows a sample-dependent tangent-space dimension.

Can the authors explain why the mean x0x_0...

Please refer to the general response.

Thank you again for your time and for your thoughtful review. As the discussion period ends soon, we wanted to kindly check whether our response has addressed your concerns. If there are any remaining questions or points that would benefit from further clarification, we would be glad to address them before the deadline.

Dear reviewer,

Did the authors address your concerns in their replies?

Reviewer_wpNsOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper attempts to theoretically explain adversarial robustness via a sensitivity field. Specifically, the paper defines the sensitivity field of a given classifier as the classifier's gradient vector field, then observes that a differentiable classifier should yield a conservative sensitivity field. The paper further defines a robust sensitivity field with a tangency condition. By projecting an existing sensitivity field to meet the tangency condition, the paper constructs a robust and accurate classifier as a differentiable function whose gradient is the robust sensitivity field. Further theoretical analysis shows that, depending on the choice of robust sensitivity field, such a differentiable, robust, and accurate classifier may not exist. To test this idea in a multi-class setting, the paper proposes a function defined by the path integral of the sensitivity field. The paper validates this idea through experiments on adversarially trained sensitivity fields.

Critical strengths
  1. The proposed idea is novel and interesting. Attempts to construct a solid theoretical foundation for adversarial examples are highly encouraged.
Critical weaknesses
  1. The paper does not provide the practical value needed for adversarial example research. The proposed idea of learning a sensitivity field and using path integrals to define the classifier introduces unacceptable computational cost in practice. The theoretical findings might be interesting; however, it is unclear how to apply them in practice. The overall lack of practicality significantly reduces the paper’s value.
  2. The experiments are limited. First, the paper considers only adversarially trained sensitivity fields, whereas sensitivity fields from vanilla training have measurable properties, such as conservativeness and a lack of the tangency condition. Some experiments used too few per-batch samples to generalize. The paper reports poor results (e.g., low p-values) that may violate the desired properties, but such results should not appear in such simple datasets.
Questions for authors
  1. Please improve the paper's overall presentation.
    • Summarize related work and add a dedicated section comparing the paper’s position to the existing work.
    • If possible, add more illustrative figures that describe the proposed theoretical concepts.
    • The paper lacks sufficient detail on the sensitivity field learning mechanism, e.g., the internal neural network architecture and training loss functions, to reproduce the results. Describe thoroughly how to construct the proposed idea.
  2. The overall experiments should be improved.
    • The adversarially trained sensitivity fields lack a baseline for comparison. A natural baseline would be the sensitivity fields of an accurate but non-robust model. Consider adding baseline measurements to compare.
    • 10 batches of 32 examples seem too small to draw generalizable conclusions. Increase the scale of the experiments that used a small number of samples.
    • The described Euler approximation is too simple and would be inaccurate when xx is far from x0x_0. While adversarial examples tend to be close to xx, we cannot guarantee that xx is close to x0x_0.
    • Using the centroid point for x0x_0 clearly degrades the robustness accuracy. This suggests the proposed method may not be reliable enough (even with high computational cost) in practice.
    • In Section 5.2, please justify the reason why conservativeness can be violated in some cases.
    • Approximating the tangent space with nearest neighbors is inaccurate on most image datasets because the samples are generally sparse and do not reflect the local geometry well enough. MNIST is an exceptional case because the underlying data vary little relative to the number of samples. The failures in Table 4 might simply reflect tangent-space estimation failures due to data sparsity rather than the tangency that the paper tries to show.
  3. The paper proposes to use neural networks to approximate the sensitivity field. However, the underlying issue is that we are now using an architecture (e.g., neural networks) that we already know may fail to learn a robust function, i.e., adversarial examples exist. Why should we believe such a defect wouldn't happen when fitting a vector field, which seems like a more complicated task?
  4. The idea requires a function whose zero set is the data manifold. Does any data manifold have such a function? If not, provide the condition for the function's existence.
    • If the manifold’s connectedness (assumed in Theorem 1) is necessary, this is a serious flaw because classification questions separate distinct connected components, so data manifolds are not connected in general.
    • The paper identifies curvature information as one source of the obstructions. If the true data manifold is disconnected but the zero set of ϕ\phi is connected, some part of the zero set would not lie on the true data manifold. High curvature in those areas should not contribute to the obstructions.
    • The counterexample in Theorem 3.4 shows that it is not possible to learn a classifier function that fluctuates over a circle while its gradient field remains ϕ\phi-strongly tangent (which I expect the gradient vectors to be orthogonal to the circle). However, in practice, there should be disconnected parts across two or more classes, and the function value may fluctuate in the gaps between connected components. If the authors consider two or more classes to coexist in a single connected component, then the class distribution must change somewhere, and robustness cannot be achieved at the class transition.
7 comments
AuthorsOpenReview ↗

Response (1/4)

Summarize related work and add a dedicated section comparing the paper's position to the existing work.

We have added an appendix as described in the general response.

If possible, add more illustrative figures that describe the proposed theoretical concepts.

We have added a figure showing a sensitivity field that is strongly tangent to S1S^1 to help the reader visualize the concept.

The paper lacks sufficient detail on the sensitivity field learning mechanism...

The network architectures and the hyperparameters for training and attacks are given in Appendix B, and the construction of the branches is described in lines 306--313. To facilitate reproduction of the results, we will bring all of this information together in the appendix.

The adversarially trained sensitivity fields lack a baseline for compariso...

The main purpose of the paper is to compare conservative and non-conservative fields, and the conservative fields serve as the baseline in this comparison.

In line with your suggestion, we have additionally evaluated sensitivity fields trained without adversarial training, in order to isolate the effect of adversarial training. The results (mean ±\pm standard deviation over the 10 batches) are added to the appendix for reference.

On MNIST and Fashion-MNIST, the fields trained without adversarial training are accurate but not robust, as expected of such a baseline. On CIFAR-10, the field retains a considerable part of its accuracy under attack even without adversarial training. We believe this is because a budget of 8/2558/255 is small under oracle paths: the perturbation adds only a short segment x↦x~x\mapsto\tilde{x} to the path, and the first-order response of the baseline sensitivity field along such a segment is already substantially robust. The same budget, however, is large enough to break the adversarially trained conservative field, whose accuracy drops from 74±7%74\pm7\% (clean) to 28±7%28\pm7\% (adversarial), whereas the adversarially trained non-conservative field attains 74±10%74\pm10\% and 70±9%70\pm9\%, respectively.

We note that the standard perturbation budgets are calibrated for conventional, conservative models. A meaningful comparison among non-conservative fields, such as between those trained with and without adversarial training, would require new standards, and the paper makes no claims about the relative merits of different non-conservative methods. Its comparison is between conservative and non-conservative fields.

This additional baseline therefore does not change the main conclusions already reported: non-conservative fields show substantial improvements over conservative fields in generalization to unseen attacks, better alignment with the tangent space, and a meaningful deviation from conservative sensitivity fields.

10 batches of 32 examples seem too small...

As explained in the general response, sample size alone is not a sufficient basis for assessing a statistical test. The experiments report pp-values wherever statistical testing is applicable. The only experiment that does not report a pp-value is the second one, where robust accuracy was the more important statistic.

Running the full AutoAttack suite on the entire test set with the precise adaptive-step integrator would take days of computation. Given the size of the observed effect relative to its variance, we do not expect a larger sample under the precise integrator to change the conclusion. In particular, Welch's tt-test, computed over the 10 per-batch robust accuracies, gives p≤10−6p\leq 10^{-6} for the null hypothesis that the robust accuracy of the conservative field is greater than or equal to that of the non-conservative field.

Considering also the consistency of the results across all three datasets, we believe that the sample sizes are sufficient for the comparisons that these experiments are designed to make.

The described Euler approximation is too simple...

This experiment tests the first-order response of the model along the path x↦x~x\mapsto\tilde{x}. The backward step is used to ensure that the vector field does not vanish during training, while providing the bb term for h(x~∣x↦x~,b).h(\tilde{x}\mid x\mapsto\tilde{x},b).

The precision of the path-integral computation along x0↦xx_0\mapsto x is not what is being examined here. The experiment shows that the sensitivity field at xx is sensitive to meaningful directions while remaining insensitive to adversarial directions.

We would also like to clarify that the sensitivity fields in the first and second experiments are not the same: each is trained and tested separately with its own approximation method. The remaining experiments use the sensitivity fields trained in the second experiment.

AuthorsOpenReview ↗

Response (2/4)

Using the centroid point for x0x_0 clearly degrades...

The experiments are not designed to test the practicality of the method; the use of oracle paths already makes this explicit. The paper presents a theory, and the experiments are designed to test that theory.

That said, the purpose of the ablation study on x0x_0 is precisely to show that the choice of paths during training has an observable and substantial effect on the outcome. In other words, the paper demonstrates that path design is a crucial part of the model and not an arbitrary choice.

We believe that this observation is important and informative about path design. We have therefore detailed our geometric intuition in the general response and will add this discussion to the main text.

In Section 5.2, please justify the reason why conservativeness can be violated in some cases.

We assume that the question concerns why non-conservativeness is not observed in every case, since the null hypothesis of the experiment is vanishing circulation.

As mentioned in lines 322-323, vanishing circulation over the particular loops tested does not establish that a vector field is conservative: for that, the circulation must vanish over every closed loop in the domain. The cases in which the null hypothesis is not rejected therefore do not show that the field is conservative. They only indicate that, for that particular sensitivity field, the branch obtained from straight paths agrees with the branch obtained from oracle paths for a small percentage of tested loops.

Approximating the tangent space with nearest neighbors is inaccurate...

We have also performed the experiment with the tangent space approximated using image invariants, inspired by Tangent Prop (Simard et al., 1991). This experiment produces similar results, and we will add it to the appendix.

This method can approximate the tangent space accurately when a substantial bank of transformations is available. However, the universally applicable transformations (x-translation, y-translation, rotation, scaling, axis-aligned shear, and diagonal shear) form a relatively small set and fix the dimension of the estimated tangent space to six for all datasets, irrespective of the individual sample.

The PCA method, on the other hand, allows the dimension of the estimated tangent space to vary from sample to sample, although its accuracy is limited by the availability of data. The similar results obtained with the image-invariant method therefore provide additional evidence that our observations are not merely artifacts of the nearest-neighbor approximation.

The paper proposes to use neural networks to approximate...

We can interpret your question in two ways:

  • How can we know that vector fields will not exhibit other defects?
  • How can we know that the optimal vector field can be approximated by ANNs?

The answer to the first question is that we should not assume that vector fields are free from other defects. Doing so would repeat the mistake of concluding, from universal approximation theorems alone, that scalar ANNs are free from defects. It is entirely possible that another defect will be identified for vector fields. In that case, we may need to move to a hypothesis space larger than that of vector fields.

One possible direction is to investigate higher-order tensor fields and their associated integral representations. In particular, differential forms provide a natural mathematical framework for extending path integrals to surface integrals and to higher-dimensional domains of integration. Such a hierarchy would involve increasingly general geometric objects and violation of their corresponding integrability conditions.

Our intuition is that this question is best approached through the lens of explainability, and specifically the Integrated Gradients method. The relationship between integrability conditions, higher-order representations, and explainability could prove fruitful. Nevertheless, this issue is outside the scope of the current paper, and we are not in a position to draw further conclusions.

The second question can be answered directly. The defect is not necessarily a shortcoming of ANNs, but rather a limitation of representing the hypothesis as a scalar function. Universal approximation theorems establish that neural networks with suitable architectures and activation functions can approximate continuous vector fields uniformly on compact domains to arbitrary precision. Therefore, provided that the optimal sensitivity field is sufficiently regular, it can be approximated by a neural network without the conservativeness constraint that comes with representing it as the gradient of a scalar potential.

AuthorsOpenReview ↗

Response (3/4)

The idea requires a function whose zero set is the data manifold...

Regarding the representability of manifolds, the Nash-Tognoli theorem establishes that every compact smooth manifold is diffeomorphic to a nonsingular real algebraic set. Furthermore, the Whitney embedding theorem states that every smooth nn-dimensional manifold can be smoothly embedded into R2n\mathbb{R}^{2n}.

These results establish that a broad class of smooth manifolds admits algebraic representations or embeddings in Euclidean spaces. However, they do not by themselves guarantee that every particular embedding can be represented as the zero set of a globally defined regular map with exactly the required number of equations. That stronger requirement depends on the embedding and its defining functions.

The answer therefore depends on whether we are comfortable with any embedding or are we looking for a particular one. The main relevant characteristics of ϕ\phi are smoothness and dimension.

Nevertheless, the framework can represent a broad class of manifolds of practical and mathematical interest, including spheres and tori. Manifolds such as Klein bottles, real projective spaces, and Grassmannians can also be represented as zero sets, provided that an appropriate embedding and defining function are chosen; for non-orientable manifolds, this requires more defining equations than the codimension, with JϕJ_\phi of rank equal to the codimension rather than of full row rank.

In conclusion, as the paper makes explicit, the manifold assumption is a simplifying assumption which enables our geometric intuition. The true solution to your concerns is to generalize the discussion to arbitrary data distributions which is out of the scope of the paper. An intuitive direction for this generalization is to let ϕ(x)=∇log⁡p(x)\phi(x)=\nabla\log p(x), which vanishes on the critical points of pp and then use Hp(x)H_p(x) to construct the projection. However, this approach is not an incremental improvement over the paper since it is not clear how to deal with local minima or saddle points of pp. We have added this discussion to the conclusion of the paper.

If the manifold's connectedness (assumed in Theorem 1) is necessary...

Connectedness is required in Theorem 3.1 only to guarantee the existence of a path between the relevant points. Since manifolds are locally path-connected, their connected components are path-connected.

Therefore, if the manifold has several connected components, the path is simply chosen within the component containing the relevant samples. The theorem can thus be applied to each connected component independently, without requiring the entire manifold to be connected.

The paper identifies curvature information as one sou...

The situation described here differs from the assumption made in the paper. The paper assumes that

\= \\{x\in\mathbb{X}\mid\phi(x)=0\\}.$$ whereas the situation you describe corresponds to $$\mathcal{X} \subset \\{x\in\mathbb{X}\mid\phi(x)=0\\}.$$ Under the equality assumed in the paper, every point of the zero set belongs to the modeled manifold. A more general reading of the question is whether the manifold has to be connected. The answer is no. For example, consider $$\phi(x,y) \= \left((x-2)^2+y^2-1\right) \left((x+2)^2+y^2-1\right).$$ Its zero set consists of two disjoint circles. The same projection construction applies, and Theorem 3.1 can be applied to each connected component independently.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response 1/3

Practicality and Computational Burden

The method used in the paper to implement a multi-valued hypothesis is indeed computationally intensive. We chose it because it makes the sensitivity field explicit in the computation. As stated in Sections 4 and 6, this method serves to test the theory; we do not propose it as a practical way of building multi-valued models. To put the practicality of the proposal into perspective, we show below that purification methods such as DiffPure (Nie et. al. 2022) implicitly compute a path integral of a non-conservative sensitivity field.

Purification: A purification method first removes adversarial perturbations from the input and then applies an accurate but brittle classifier to the purified sample. Let π:Rd→Rd\pi:\mathbb{R}^d\to\mathbb{R}^d be the purifier and ψ:Rd→Rk\psi:\mathbb{R}^d\to\mathbb{R}^k the logits of an accurate classifier. The resulting hypothesis is h(x)=arg⁡max⁡ ψ(π(x)).h(x) = \arg\max \, \psi\bigl(\pi(x)\bigr).

An idealized purifier: Let ϕ:Rd→Rl\phi:\mathbb{R}^d\to\mathbb{R}^l be a smooth defining function of the manifold of benign samples, X=x∣ϕ(x)=0\mathcal{X}=\\{x\mid\phi(x)=0\\}, with full-rank Jacobian JϕJ_\phi near X\mathcal{X}. We take ϕ\phi to be the (vector) distance function to X\mathcal{X}. Consider the closest-point projection π(x)=arg⁡min⁡x′∥x−x′∥subject toϕ(x′)=0,\pi(x) = \arg\min_{x'} \|x-x'\| \quad \text{subject to} \quad \phi(x')=0, which is well defined in a tubular neighbourhood of X\mathcal{X}. DiffPure approximates such a map: for small diffusion times, the reverse process denoises xx by approximately x+σ2∇log⁡pσ(x)x+\sigma^2\nabla\log p_\sigma(x), and when the data concentrate near X\mathcal{X} this score points along the normal direction toward X\mathcal{X}. For a stochastic purifier, the argument below applies to each realization, or to its deterministic probability-flow counterpart.

Purification as a path integral: Define the projected sensitivity field F=Pϕ∇ψF = P_\phi \nabla\psi. Fix a base point x0∈Xx_0\in\mathcal{X} and, for each xx, the path γx:x0↦π(x)↦x\gamma_x : x_0\mapsto\pi(x)\mapsto x, whose first segment lies in X\mathcal{X} and whose second segment is the normal segment from π(x)\pi(x) to xx. On the first segment the velocity is tangent to X\mathcal{X}, and this segment contributes ψ(π(x))−ψ(x0)\psi(\pi(x))-\psi(x_0) according to Theorem 1. Since ϕ\phi is a distance function, the second segment is an integral curve of the rows of JϕJ_\phi; there Pϕγ˙=0P_\phi\dot\gamma=0, so it contributes nothing. Hence ψ(x0)+∫γxF⋅dγ=ψ(π(x)),\psi(x_0) + \int_{\gamma_x} F \cdot d\gamma = \psi\bigl(\pi(x)\bigr), and purification is exactly the branch of the multi-valued hypothesis induced by FF that is selected by the path γx\gamma_x: first along the data manifold, then along its normal.

A purification method can also be multi-valued in a manner that is tightly connected to the topology of the manifold of benign samples. The minimization defining π(x)\pi(x) may have several solutions rather than one, in which case h(x)h(x) takes several values. For example, if X=S1\mathcal{X}=S^1, every point of S1S^1 is a nearest point to the origin. The corresponding set of classifier values at the origin is therefore ψ(z)∣z∈S1.\\{\psi(z)\mid z\in S^1\\}.

This exposition is too informal for the main paper, so we will place it in the appendix. Nevertheless, it gives the reader a better intuition for the possibilities that the framework provides. It is also informative about which purification methods can be effective. For example, if both ψ\psi and π\pi are differentiable, then their composition ψ∘π\psi\circ\pi is differentiable as well. Such a purifier therefore does not remove the differentiability constraint that motivates our framework.

In conclusion, the method we use to test our hypothesis is not indicative of all the ways in which the proposal could be exploited. Our proposal is to represent the sensitivity field directly, and the explicit path-integral formulation is one way to harness this representation in a controlled manner. The other practical methods we are aware of do not isolate the effects that we want to observe as clearly.

AuthorsOpenReview ↗

General Response 2/3

Sample Size of the Experiments

Reviewers have raised concerns about the sample sizes of the experiments. The second and the last experiments used a sample size of 320. However, sample size alone is not a sufficient indicator of the soundness of a statistical test, which is why the paper also reports statistical significance and, where applicable, effect sizes.

Second Experiment: For the second experiment, the differences between the robust accuracies of the conservative and non-conservative vector fields are large enough that a sample size of 320 is informative about what the experiment is designed to show: after adversarial training, non-conservative vector fields generalize better to unseen attacks than conservative ones. To quantify how little data this conclusion requires, we additionally use a betting-based any-time valid test (Waudby-Smith and Ramdas, 2020), whose implementation we verified against ConfSeq v0.0.11 (Howard et al., 2019).

Both models are evaluated on the same test images, and each is attacked with its own AutoAttack (w/o Square attack) adversarial example. For the ii-th image we define the paired difference

- \mathbf{1}\left[\text{conservative correct}\right] \in \\{-1, 0, 1\\},$$ and test $H_0\colon \mathbb{E}[D] \le q$ against $H_1\colon \mathbb{E}[D] > q$ with the e-process \begin{equation} K_t(q) = \prod_{i=1}^{t} \bigl(1 + \lambda_i \, (D_i - q)\bigr), \end{equation} where each bet $\lambda_i \in \bigl[0, \tfrac{1}{2(1+q)}\bigr]$ depends only on $D_1, \dots, D_{i-1}$. Test images are visited in a random order fixed in advance. Under $H_0$, $K_t(q)$ is a nonnegative supermartingale, so by Ville's inequality, rejecting $H_0$ as soon as $K_t(q) \ge 1/\alpha$ controls the type-I error at level $\alpha$ under continuous monitoring and optional stopping. We use $\alpha = 0.05$. For $q = 0$, the null hypothesis is rejected after at most 21 samples (not batches): after 13 samples on MNIST and FMNIST and after 21 samples on CIFAR-10 (Table 1). Continuing the e-process over the remaining samples of the first batch ($n = 32$) and inverting the test over $q$ gives any-time valid 95\% lower confidence bounds on the robust-accuracy gap $\mathbb{E}[D]$. In other words, with probability at least $95\%$, the true robust-accuracy gap $\mathbb{E}[D]$ lies above the lower confidence bound (LCB) simultaneously at every sample size $t$. Hence the bound reported at $n = 32$ remains valid even though the stopping point could have been chosen after inspecting the data, and the experiment could be continued indefinitely without invalidating it. **Table 1: Any-time valid comparison of robust accuracy between the non-conservative (NC) and conservative (C) vector fields at $\alpha = 0.05$.** *Left block:* samples needed to reject $H_0\colon \mathbb{E}[D] \le 0$, with the e-value at rejection. *Right block:* the first batch ($n = 32$) with each model's robust accuracy, the observed gap, and the 95% any-time valid lower confidence bound (LCB) on the gap. | | Stopping at $q = 0$ | | First batch ($n = 32$) | | | | |----------|------:|------:|-------:|-------:|---------:|---------:| | **Dataset** | **$n$** | **$K_n$** | **C (%)** | **NC (%)** | **Gap (pp)** | **LCB (pp)** | | MNIST | 13 | 25.6 | 0.0 | 65.6 | 65.6 | 33.4 | | FMNIST | 13 | 28.2 | 3.1 | 53.1 | 50.0 | 19.3 | | CIFAR-10 | 21 | 20.5 | 40.6 | 81.2 | 40.6 | 10.1 | As mentioned in the paper (lines 372--375), a comparison of the approximation capacities of the two classes of vector fields cannot favor the conservative class, since conservative fields form a strict subset of all vector fields. Such a comparison would therefore not be informative. **Last Experiment:** The last experiment investigates the non-conservativeness of the projected vector fields by measuring the norm of the antisymmetric part of their Jacobians. The statistical test examines the null hypothesis that this norm vanishes, and the resulting $p$-value provides strong evidence against it. Note that $\phi$ is not learned in this experiment and that the antisymmetric part of the Jacobian of $\nabla\psi$ vanishes up to machine precision. The details of the experiment in Appendix B.1 show that the observed effects are too large to be attributed to numerical error.
AuthorsOpenReview ↗

General Response 3/3

Connection to the Literature

The paper discusses the literature that directly contributes to the framework in lines 159--185. We have added a detailed literature review to the appendix, covering:

  • The geometric and causal accounts (Tanay & Griffin, 2016; Tang et al., 2021), which inspired the basic setup of our proposal;
  • Other explanations of the accuracy--robustness trade-off (Tsipras et al., 2019; Haldar et al., 2024);
  • Results on adversarial examples in random networks (Bubeck et al., 2021; Bartlett et al., 2021; Montanari & Wu, 2023), as requested by Reviewer gQYq, together with their relation to our framework. \end{itemize}

Choice of x0x_0

The effect of the choice of x0x_0 in the experiments has intrigued both us and the reviewers. In particular, training path-dependent hypotheses using straight paths starting from x0=0x_0=0 produces substantially better robustness than using the mean of the training set, Xˉ\bar X, as the starting point.

We do not have a theoretical explanation for this observation. Nevertheless, we believe that a geometric analogy provides some intuition. Consider the unit circle S1S^1. Its center of gravity is the origin, and every straight path from the center to a point on S1S^1 is a radius. Such paths intersect the circle at right angles. Now consider a path integral along one of these radial paths for a sensitivity field that is strongly tangent to S1S^1, as in Theorem 3.4. Since the radial direction is normal to the circle and the sensitivity field is tangent to it, their inner product vanishes. Consequently, the integral along every radial path vanishes, irrespective of the particular tangential field, and the resulting branch is constant. It therefore cannot represent a non-constant classifier on the circle.

Making this branch accurate would require the sensitivity field to have a component in the radial direction, which would move it away from strong tangency to S1S^1. In other words, the center of gravity of the manifold can be one of the worst choices for x0x_0 when constructing a branch from straight paths.

We use this example as an analogy for the experimental observation. Note that, in the analogy, the center of the circle plays the role of the data mean Xˉ\bar X, not of the choice x0=0x_0=0 in the experiments. The mean of the training set can be an off-manifold point, and straight paths from such a point to samples on the manifold may run in directions that are approximately normal to the manifold. The origin of the input space, on the other hand, is in general not the centroid of the data and induces a different family of paths. The experiment shows that this choice has a substantial effect on the robustness of the learned path-dependent hypothesis.

We have added this discussion to the main text of the paper, as we believe it is an important observation about path design.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.