acceptodds
Under review as a conference paper at ICLR 2027

Characterising Input Loss Curvature Under Backdoored Networks

Abstract

Backdoor attacks, introduced through data poisoning or architectural manipulation, induce attacker-chosen behaviour on inputs carrying a trigger while leaving clean model performance essentially unchanged. The eigenvalues of a model's input loss Hessian, , measure the local curvature of the loss surface around an input and may therefore contain information about triggers in backdoored networks. We study this through the exact decomposition , where is the generalised Gauss-Newton term and is a model-curvature correction. For cross-entropy classification, we derive bounds showing that output-space curvature collapses as prediction confidence approaches unity. In locally affine networks, almost everywhere, so input curvature also collapses if the model Jacobian remains bounded. We further show that locally affine additive backdoors only change , whereas multiplicative composition introduces an indefinite residual term of rank at most two. We validate these results experimentally using ResNet-18 image classification models with ReLU and SiLU activations. For additive backdoors, we observe eigenvalue collapse under ReLU activations that does not persist when SiLU is used, and multiplicative backdoors produce a large model-curvature correction. Overall, these results show that input curvature depends jointly on backdoor construction, model confidence, model architecture, and loss function.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safety, Security, Backdoors, Hessian, Loss Landscape, Image Classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_G1ibOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how backdoor attacks affect the input-space loss curvature of neural networks, measured through the eigenvalue spectrum of the input Hessian. Its central goal is to understand whether observed curvature changes are intrinsic signatures of backdoors or consequences of other factors, such as prediction confidence, model architecture, and loss functions. By studying three backdoor mechanisms- data poisoning, additive architectural backdoors, and multiplicative architectural backdoors- on ResNet classifiers with ReLU and SiLU activations, the theoretical findings are well supported: Input-Hessian curvature is not a universal or uniquely identifiable signature of backdoors.

Critical strengths
  1. The paper applies the Hessian decomposition to backdoored networks, showing that locally affine models have R=0, whereas nonlinear models can exhibit substantial residual curvature.

  2. The authors show that additive architectural backdoors preserve zero residual curvature under local affinity, while multiplicative backdoors can introduce an indefinite, rank-at-most-two curvature correction, producing qualitatively different Hessian spectra.

  3. Experiments on CIFAR-10 and additional image-classification settings confirm that ReLU backdoors often exhibit confidence-driven curvature collapse, whereas smooth activations and multiplicative backdoors can retain or introduce substantial curvature. This strengthens confidence in the paper's central conclusions and supports its value as a foundational analysis of input-loss curvature.

Critical weaknesses
  1. The empirical evaluation mainly considers image classifiers with relatively simple backdoor mechanisms. However, it is unclear whether the findings extend to more advanced, stealthier backdoor triggers beyond simple patch triggers and to modern architectures such as Transformers.

  2. Although the paper shows that input curvature is not necessarily a backdoor-specific signature, it does not establish whether the proposed decomposition provides some insights for backdoor detection or attribution.

  3. Although the paper attributes curvature collapse under successful backdoors mainly to prediction confidence, it does not systematically compare confidently correct predictions on clean inputs with confidently incorrect, backdoor-target predictions on triggered inputs under matched confidence levels. Such a comparison, ideally including paired clean/triggered inputs and separate analyses of the (G) and (R) terms, would help establish whether the observed curvature changes are specific to backdoor activation or reflect general properties of high-confidence predictions. Without these controls, the empirical evidence for the proposed mechanistic interpretation remains somewhat incomplete.

  4. The abstract is somewhat overly mathematical, which reduces its accessibility and clarity. In addition, the introduction could benefit from a more coherent motivation and narrative. It begins with a general discussion of the input-Hessian spectrum before establishing its relevance to backdoored neural networks, making the transition to the paper's central research question somewhat abrupt.

Questions for authors

None

3 comments
AuthorsOpenReview ↗

Extra Work

Thank you for taking the time and energy to review this work, the comments provided will substantially improve this work.

Extra Experiment

Thank you for the suggestion, we agree that the paper would benefit from this type of confidence matched experiment.

From the theoretical results, we can establish that when using CE loss, the G term is label-independent, and the curvature approaches zero as prediction confidence increases (Corollary 3). This applies to both confidently correct and incorrect predictions. This type of experiment would be particularly informative when R is not zero due to its dependence on the evaluation label. We investigate the separate contributions from G/R in some of the experiments. For example, in Figure 3, we evaluate the same triggered images under clean and backdoor labels to demonstrate the label-dependence in R. Further, in Section 4.3 we compare paired clean/triggered inputs and separate G and R.

Abstract

We will edit the abstract to replace some complicated mathematical phrases with more understandable versions. We propose the following replacements:

  1. Old: that output-space curvature collapses as prediction confidence approaches unity.

New: that loss curvature collapses as prediction confidence increases.

  1. Old: We further show that locally affine additive backdoors only change G, whereas multiplicative composition introduces an indefinite residual term of rank at most two.

New: We further show that locally affine additive backdoors only change G, whereas multiplicative composition introduces an R term with two eigenvalues (one positive and one negative).

Please do let us know if there are further changes that would like us to make to the abstract to improve its clarity.

Introduction

Regarding the comments on the introduction, we have tried to write something that gives the necessary background in both parameter and input Hessians and their uses, as well as backdoors in neural networks, followed by recent literature on backdoor signatures in input Hessians, ending with our research question and what we contribute. We’d be grateful if the reviewer could expand upon which part they consider to be abrupt so that we can make changes to their satisfaction.

Thank you for the detailed responses and clarifications. I appreciate the authors' efforts to address my concerns. I understand the intended scope of the work and agree that extending the experiments to more sophisticated backdoors and Transformer architectures could be left for future research. The clarification regarding the practical implications of the proposed decomposition is also helpful, and the suggested revisions to the abstract should improve its accessibility. However, my main concern regarding the empirical validation remains insufficiently addressed. While the theoretical analysis establishes a relationship between prediction confidence and input curvature, and Figure 3 provides evidence of the label dependence of the residual term, these results do not fully replace the requested confidence-matched comparison between clean and triggered inputs. Concretely, even at matched prediction confidence, differences in the model Jacobian and output distribution may lead to different curvature behavior. Such a controlled comparison, ideally with separate analyses of the G and R terms, would help establish whether the observed curvature changes are primarily driven by prediction confidence or reflect additional effects associated with backdoor activation. I therefore encourage the authors to strengthen the empirical validation or explicitly acknowledge this limitation in their conclusions.

Thank you for your continued engagement in this discussion.

We agree that the paper would benefit from including these plots. While space constraints prevent us from including them in the main text, we would be happy to include them in the appendix.

Specifically, we propose including plots of the eigenvalues of H, G, and R across different confidence bands, considering both piecewise-affine and non-piecewise-affine models, as well as different types of backdoors.

Please let us know if there are any additional aspects of your comments that you feel this proposal does not address.

Reviewer_8kNZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how backdoor attacks affect the input loss Hessian spectrum and whether curvature changes provide backdoor-specific signals. The key idea is to decompose the input Hessian into a Generalised Gauss-Newton term (G) and a model-curvature correction (R), allowing the authors to distinguish the effects of prediction confidence, model nonlinearity, and backdoor construction. Theoretical and empirical results show that high-confidence predictions can cause Hessian eigenvalues to collapse in locally affine networks, while multiplicative backdoors or smooth activations can introduce substantial additional curvature. Overall, the paper demonstrates that similar backdoor behaviours can produce very different Hessian spectra, highlighting that curvature is not a universal signature of backdoor attacks.

Critical strengths
  • The paper addresses an interesting question of how to interpret input Hessian spectra under backdoor attacks.

  • The paper derives quantitative bounds connecting prediction confidence to Hessian curvature and characterises the different effects of additive and multiplicative backdoors.

  • The experiments consider different backdoor mechanisms, activation functions, loss functions, and model architectures.

Critical weaknesses
  • Although the paper explains why different backdoors produce different curvature patterns, it does not demonstrate how these findings can improve backdoor detection or defense.

  • The experiments primarily focus on image classification with simple patch triggers and explicitly constructed architectural backdoors.

Questions for authors

Thank you for submitting the interesting work to ICLR 2027. I still have a major concern.

The paper provides meaningful theoretical insights into how different backdoor mechanisms affect the input Hessian spectrum. However, the current work is primarily focused on theoretical characterisation, and its practical implications remain underexplored. It would be valuable for the authors to discuss how these findings could be extended to more realistic security scenarios. For example, could the identified curvature patterns be exploited by attackers to evade Hessian-based detection methods, or could the decomposition of the Hessian provide useful signals for developing more robust backdoor detection or defense techniques? Even if implementing such applications is beyond the scope of this paper, a more detailed discussion of these potential directions and their associated challenges would help strengthen the practical significance of the work.

4 comments
AuthorsOpenReview ↗

Practical uses and possible extensions

Thank you for taking the time and energy to review this work, the points made are relevant and will help to improve this submission.

Please see the official comments to all reviewers for responses to the practical uses and possible extensions as this was brought up by multiple reviewers.

Hi,

Thanks for your rebuttal. I still think the empirical experiment is necessary, especially for the security field. It could even be more important than theoretical findings because we finally want to evaluate the security risks regarding a real system and make it robust. The theoretical findings could be very useful guidance for this purpose. In my point of view, theoretical findings alone is not the complete story.

Therefore, I tend to keep my current rating.

Best

In addition, the authors also mentioned "for example by training a data poisoning with an additional term in the loss function which increases the top eigenvalue." in the rebuttal. Actually, if this is true, this could be another limitation, because the attacker can evade the proposed method (if we consider it as a defense or possibly defense methods based on the theoretical findings in the paper). This is similar to the idea of adaptive evaluation [1]. In other words, the authors are also aware that the practical value is limited.

[1] On Adaptive Attacks to Adversarial Example Defenses, NeurIPS 2020.

Thank you replying to our comments. We agree that a proposed security defence would require substantial practical evaluation, including evaluation against adaptive attacks. However, we would like to clarify that our paper proposes neither a detector nor a defence. Its contribution is to explain, theoretically and experimentally, why input-Hessian spectra can differ substantially between backdoor mechanisms, and why observed curvature should not be interpreted as a backdoor-specific signature without controlling for confidence, model geometry, and loss.

Our response about the curvature-aware poisoning was intended as a possible research direction, rather than an experimentally established evasion attack, as brought up during the review process. We acknowledge that “it would be possible” was too strong: demonstrating such evasion would require specifying a detector and showing that an adaptive attack defeats it while preserving clean performance and attack success. We will revise this wording accordingly.

Could you clarify whether your remaining concern is that additional experiments are needed to support a particular claim made in the paper, or that the characterisation contribution has insufficient significance without a demonstrated detection or defence application? This would help us address your concern directly.

Reviewer_KJEUOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This work investigates changes in the input loss curvature in simple backdoored networks for image classification and multiple changing circumstances like: backdoor construction (data poisoning and architectural backdoors - additive and multiplicative - describing the type and how big of a change backdoor introduce to the model), model architectures and losses.

As most important contribution I would point theoretical derivation and prediction of factors resulting in curvature changes for clean/triggered data in different models under those circumstances. It definitely does not look simple and easy, due to various factors present at the same time.

Empirical experiments looks like only of secondary importance to the Authors, ant they are also not straight-forward to interpret given current explanations.

Critical strengths
  1. Theoretical derivation of factors crucial for input loss curvature and how they interact with each other.
  2. Predictions validated (or at least looks like it, but difficult to interpret) on small image classification networks like ResNet-18
  3. Interesting discussion and potential important insights for backdoor detection, and why some methods of detection are overconfident in the backdoor signal based on curvature that can be connected with different issues, since it is not straightforward signal.
Critical weaknesses

W1. The work was interesting and partially novel for me in terms of informativeness, but (maybe due to lacking background to understand easily the theory and given the review timeframe/ or presentation issues) very hard to understand. In particular, it was hard to interpret results and consequences of claims and derivations as a big picture. There were a lot of changing parts and details, the work would really improve if the figure or table would sum up all the cases that are changing, their definitions, and the results plus interpretation. I think it may be a fixable issue, but the problem remains, that unfortunately I cannot confidently assess the significance of this work. W2 Additional discussion of what are the consequences of those results for existing backdoor and data poisoning defenses would be beneficial. W3 Much more space is needed to explain figures, why authors measures those values/ cases, and how to interpret obtained results.

Questions for authors

See weaknesses. Additionally:

  1. I have a question about how authors see backdoor training-time, training-loss attacks like blending - adding additional backdoor term to loss during training. If I understand properly the notion of architectural backdoor (correct me if I am wrong here), this scenario is somewhere between data poisoning backdoors and architectural backdoors, would definitely somehow impact the loss curvature, yet it may be much harder to predict, since the loss may be more controlled by the attacker during training. Have the authors consider investigating those types of backdoors?
  2. W3 -> I do not understand right panels of some plots, where for part of poisoned data 'x's we see correlation, and there is a group being in a totally different cluster uncorrelated.
3 comments
AuthorsOpenReview ↗

Clarity of figures

Thank you for your review, we acknowledge your points around clarity in the cases and in the figures and we can add the table as suggested, and include further detail in the plots and measured values to help with readability.

Clarity and Figures

We agree that this would benefit from clearer exposition in the text. The amount of space we have available to do this is limited however, we will more clearly signpost in the text which subplot we are referring to at each point and include more intuition as per below.

Measured Values:

  • p(1−p)∥g∥2p(1-p)\lVert g \rVert^2 (which is simplified to p(1−p)p(1-p) in one plot: Measured since it is approximately the value of the largest eigenvalue of GG (as shown in Eq 5). We hope to show that what we see in empirical experiments match what we expect theoretically = ie the observed eigenvalues are similar to the expected approximations of GG

  • ∥J∥2\lVert J \rVert^2: Our study is mostly around the curvature, however the approximation forms also include a gradient term. We include this Jacobian metric to show the Jacobian behaviour too.

  • ρ(A)=max⁡{∣λmax⁡(A)∣,∣λmin⁡(A)∣}\rho(A) = \max \lbrace \lvert \lambda_{\max}(A) \rvert, \lvert \lambda_{\min}(A) \rvert \rbrace: The spectral radius – the largest in magnitude as for some cases we have a negative eigenvalue

  • ΔH=medianilog⁡10[ρ(Htrig,i)/ρ(Hclean,i)]\Delta_H = \mathop{\mathrm{median}}\nolimits_i \log_{10}\left[ \rho(H_{\mathrm{trig},i}) / \rho(H_{\mathrm{clean},i}) \right]: To compare the effect of triggering on the largest in magnitude eigenvalue

  • ηR=medianilog⁡10[ρ(Rtrig,i)/ρ(Htrig,i)]\eta_R = \mathop{\mathrm{median}}\nolimits_i \log_{10}\left[ \rho(R_{\mathrm{trig},i}) / \rho(H_{\mathrm{trig},i}) \right]: To compare the spectra of RR to HxH_x

Figure 1: We compare the true largest eigenvalue of HxH_x against our approximation for the largest eigenvalue of GG (5). In this first case they do fall along the y=xy=x line, illustrating that the curvature of the data hessian is as we expected, the curvature of the GG term only. This also enables us to show that the true eigenvalues are dependent on the model confidence (pp). Where the plotted values get extremely small, to assist in visualisation, we have applied some value clipping in both confidence, pp, and in eigenvalue λ\lambda. Hence the two clusters, the additive architectural points green are clipped by their eigenvalues which get extremely small, and the data-poisoned points are clipped at a high confidence. We can update this clipping plot for camera-ready version, as we appreciate these clusters are confusing.

Figure 2: (left) This is the same plot as above, and our reasons for plotting these values is the same. We can see the clean points (blue) lie along the y=xy=x line, showing that the eigenvalues as approximated by (5) are a good estimate of the true eigenvalues of HH, whereas in the backdoored case the true eigenvalues are larger than these approximated values – showing the presence of the RR term in the curvature which is activated by a misclassification caused by the triggered input and a non-linearity within the model. (right) ∥J∥\lVert J \rVert vs top eigenvalue; we show the effect of the backdoor on the gradients for comparison.

Figure 3: (left) This plot simply shows that there is a difference in curvature in this non-linear case when the model is correct or incorrect. (right) To illustrate the results in Prop 5. We plot the top 10 eigenvalue magnitudes under different configurations of the architectural backdoor to show how the top-10 eigenvalues change as the backdoor gets stronger. The top-2 get larger and larger and the remaining 8 gets smaller – which is explained by the results derived in (Prop 5).

Figure 4: The aim of section 4.3 was to isolate the effects of the GG and RR terms for similar strength backdoors. In Figure 4 we plot 3 subplots: (a) plots the prediction-change rates (percentage of predictions changed by the trigger), (b) ΔH\Delta_H which measures how triggering effects HxH_x, (c) ηR\eta_R compares spectral radii of RR and HxH_x, we look at clean models, and additive and multiplicative backdoors for ReLU and SiLU networks. We compare to data poisoning in the appendix. These results reflect what we saw in the theory section (Prop 4 and 5): that for the additive backdoor composition on a ReLU network, the triggered images cause a collapse in eigenvalues which is different to the multiplicative composition (Figure (b)). If we compare the spectra of the residual term to the full Hessian (Figure (c)), we can see how the RR term contributes when using SiLU activations and in the multiplicative backdoor composition.

AuthorsOpenReview ↗

Additional Backdoors

Additional Backdoors

While we didn’t consider this type of backdoor in this work, it would be an interesting addition! From my understanding, for a construction such as that in https://arxiv.org/abs/2005.03823, the attack introduces an additional term to the training objective: Ltotal=Lclean+λLbackdoorL_{\mathrm{total}} = L_{\mathrm{clean}} +\lambda L_{\mathrm{backdoor}}. This changes the learnt model parameters, rather than the architecture itself. Meaning that although the attacker intervenes on the training the loss, the resultant model will behave more like a poisoned model at inference time.

Since our analysis considers the input Hessian of the evaluation loss, our results will apply in this case: for a piecewise-affine model, the R term would still be zero almost everywhere and any changes to the input-loss curvature will be from changes to the model’s logits and Jacobian. Moreover, our theoretical results for CE, including the dependence on confidence will apply. However, for the non-piecewise affine models the R term could be affected by the learnt parameters and the curvature of this will be influenced by the new training loss. Adding this to the experimentation would be an interesting direction to future work. Our current manuscript would benefit from the addition of discussion of this case.

AuthorsOpenReview ↗

Practical uses and possible extensions

Please see the official comments to all reviewers for responses to the practical uses and possible extensions as this was brought up by multiple reviewers.

Comments on the paper

2 comments
AuthorsOpenReview ↗

For all reviewers - on the practical applications of this work

Firstly, we'd like to thank the reviewers for taking the time and energy to review this work and for providing insightful feedback which has helped to make this submission better. Below we have included responses to comments made by multiple reviewers.

Practical Application

With regard to the practical utility of this work, we’d like to re-iterate that we have intentionally not framed this work as a method for detecting backdoors – our findings don’t support a curvature-based detector without additional experimental design, however we feel the mathematical analysis and findings are useful. Most other work in detection is in parameter space and to the best of our knowledge, there is only one piece of work which focuses on input-space curvature under squared loss, as detailed in the introduction. We chose to extend this work for classification under CELoss, and others (Appendix E.2).

Our main argument is that the curvature needs to be interpreted within the context of the model and loss function, as similar input-output behaviours but different mechanics (e.g. loss functions / backdoor types etc.) produce different curvature. However, we can clarify a few practical points;

  • Firstly, many Hessian-based pieces of work use the generalised Gauss Newton form (our term G) as a simplification for the data/parameter Hessian. We show this is exact when the model is piecewise-affine, which supports its use in other work. However, if the model is non-linear in some way (either in the base model or through a non-linear architectural backdoor), then the true data Hessian (HxH_x) is not equal to the GGN (G) term. In these cases, the residual term R is non-zero and therefore G may not be a good approximator of HxH_x.

  • Secondly, we can say that studying the curvature through this decomposition into G and R is a useful methodology for model analysis. We suggest analysis which focuses on R, removing the curvature from G (since these terms are dependent on the prediction confidence), checking the first and second-order form of the loss function used by the model.

  • Thirdly, and most concretely, in the context of the multiplicative backdoors which introduce a non-linear behaviour in an otherwise linear model, we find spectral analysis may be most directly useful, but requires lots of prerequisite knowledge about the backdoor.

  • For the non-piecewise-affine case, we do not establish if the signal is able to discriminate between backdoored inputs, benign misclassifications, OOD inputs and adversarial inputs. We will add in a paragraph, as well as a figure to the appendix, to clarify this and highlight it as future work whilst stating the implications of this work for detection explicitly.

  • Moreover, it would be possible to evade curvature-based detection by controlling curvature in the backdooring process – for example by training a data poisoning with an additional term in the loss function which increases the top eigenvalue.

We will ensure these discussion points are clearly stated in the discussion of the final version.

AuthorsOpenReview ↗

For all reviewers - on the possible extensions to this work

Possible Extensions

We acknowledge that this work focuses on simple models and trigger types, and this was by design. We have included three different types of backdoors to illustrate variations between backdoor methodologies, but chose to focus on image classification. Unfortunately, the page limit restricts us to these cases, and we chose to focus on the theoretical analysis. We are able to make a few statements as to why changing the trigger shouldn’t have an impact on some theoretical findings, and why analysis of more complicated architecture requires further work. We appreciate we are missing an explanation along these lines, so we shall add a discussion into the discussion section of the paper.

  • Triggers: In the cases where R=0 (i.e., linear models), the trigger and backdoor will have no effect on these findings, since as we show it is the model confidence which is the only driving factor. In the cases where R ≠\neq 0, stealthier triggers and backdoors may have some effect – the effect can be seen prop 5.

  • Architectures: We haven’t addressed more complex architectures in this work, but is a clear direction for future work. The decomposition and results remain valid, but would need deriving for more complex architectures. Some work has explored the parameter Hessian of Transformer: https://arxiv.org/pdf/2410.10986, but it would be future work to derive similar for input Hessian. Transformer architectures are non-linear, and the attention mechanism introduces complexities not seen in standard classification. We hypothesis that there may be more interesting curvature behaviour because of these points but leave investigation for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.