acceptodds
Under review as a conference paper at ICLR 2027

Overtopping in LLMs: Spiking-Like Causal Control and the Limits of Static Intervention

Abstract

Unwanted behaviours can emerge without large changes in ordinary model performance, making them difficult to detect through behavioral evaluation alone. We therefore study overtopping, where intervening on a single activation channel affects behaviour across many examples, as a way to examine concentrated causal control inside the model. We ask what kind of causal control these high-leverage channels provide, how they behave under intervention, and whether they remain useful as the model continues to learn. Across five tasks and several model families, we find a recurring spiking-like pattern along two intervention dimensions: how much a channel is perturbed and when during generation the perturbation is applied. As intervention magnitude increases, behaviour can remain stable until a threshold is crossed, then switch and stay changed; across generation steps, the effect tends to be often concentrated at a particular step. We also find strong interactions among high-leverage channels and that the channels carrying high causal leverage can reorganize during learning even when ordinary task performance changes little. As a proof of concept, we introduce a rare trigger-dependent behaviour through controlled poisoning of a grammatical-acceptability task. We then ask whether overtopping measurements can identify useful intervention targets as this behaviour is learned. Channels that suppress the poisoned behaviour at one checkpoint often lose that effect after further training. However, measurements of the current model can still identify single-channel interventions that reduce the poisoned behaviour while limiting disruption to ordinary behaviour. Overtopping therefore reveals localized causal control that is threshold-like, interacting, and able to reorganize during learning. These properties explain both why such channels can provide useful intervention targets and why defenses based on fixed targets may become unreliable as learning continues.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal interventions, large language models, learning dynamics, activation interventions, threshold dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Un4aOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies overtopping in LLMs, where interventions on individual activation channels affect model behavior across many examples. It relates a channel’s causal influence to task competence, tests how channels interact and how effects change with intervention strength, and tracks whether influential channels shift during training. A controlled poisoning experiment further tests whether such channels can suppress unwanted behavior. The results show that joint interventions often weaken individual effects, behavioral changes frequently exhibit persistent thresholds, and effective intervention targets can change during training.

Critical strengths
  1. The paper provides a detailed empirical analysis of activation interventions across several tasks and models. The experiments go beyond identifying influential channels and examine their interactions, intervention strength and timing.

  2. The findings on channel interactions and changes during training are useful. In particular, the results show that combining individually effective interventions may reduce their effects, and that intervention targets identified at one checkpoint may become ineffective after further training.

  3. The paper is organized around four clearly stated research questions, and the limitations section is candid about construct, internal, and external validity. That made it easier to see what the evidence does and does not support

Critical weaknesses
  1. The evidence for a distinct spiking-like mechanism is not so convincing. Threshold-like changes in binary outputs can arise when smoothly changing internal representations cross a decision boundary. Although the authors examine logit margins and intervention timing, these results do not clearly distinguish the proposed interpretation from this simpler explanation. The finite-grid threshold correspondence also does not establish a distinct computational mechanism. The authors themselves acknowledge in Section 7 that a smooth latent trajectory can produce the same threshold pattern when it crosses a decision boundary. If so, it's unclear what the spiking analogy explains beyond that simpler account.

  2. The practical implications of the intervention results is limited. The defense experiment focuses on a single controlled poisoning setting and uses a matched clean reference model. The current-checkpoint intervention results are encouraging, but they do not establish whether the approach would work under more realistic conditions or remain effective after further training.

  3. The paper is dense and difficult to follow. There are so many terms, and they are introduced quickly. Maybe a notation table would help readers. I got lost while reading and had almost forgotten those terms by the time I reached Figure 4.

Questions for authors
  1. What evidence separates the proposed spiking-like behavior from ordinary decision-boundary effects caused by continuous changes in model activations?

  2. Do high-leverage channels exhibit threshold properties that differs from those of other channels with comparable causal strength?

  3. Can the adaptive intervention approach work without a matched clean reference model? How well does it generalize to other poisoning settings or later training checkpoints?

4 comments

Q1

We use “spiking-like” to describe measured response properties, without claiming neuronal spiking or EIF dynamics in LLMs. Just as spike initiation in the EIF model arises from smooth nonlinear dynamics, continuous changes in model activations could produce abrupt output transitions consistent with a spiking-like analogy. Smoothness therefore does not invalidate this descriptive analogy. Smoothness alone therefore does not invalidate this descriptive analogy.

Overtopping channels are identified by behavioral effects, so output changes alone is not what motivates our use of “spiking-like”, as that would be tautological. Fig. 4a shows that the largest absolute logit-margin change between consecutive intervention strengths occurs 4.80 pp more often at the persistent behavioral transition (median difference; 95% CI [2.14, 6.40]). Fig. 4b shows that the same largest absolute logit-margin change significantly increases with causal strength: in the strongest channels, the largest of ten intervention intervals accounts for 15.99% of total absolute margin change (median across conditions), versus a 10% equal-change reference. Fig. 4c uses complementary temporal sweeps to localize behavioral effects during decoding.

These specific response properties motivate our use of “spiking-like”, and are not guaranteed by construction but empirically observed in the overtopping channels population. Our interpretation, formalized in Appendix D.2, concerns this spiking-like response geometry without implying that LLMs implement EIF dynamics or an internal spiking mechanism. Whether they reflect a shared computational principle or different mechanisms remains open for future research. We will better clarify that “spiking-like” denotes these measured response properties.

Q2

At comparable high causal strength, control channels are as well overtopping high-leverage channels, regardless of whether CHA discovered them. “Comparable causal strength” means comparable held-out singleton causal reach: the fraction of direction-eligible examples flipped by full replacement, within the same run/direction condition.

We matched candidates with same-locus controls on causal strength and compared their endogenous threshold association (Appendix D, Figure 11). Across 136 matched pairs in six conditions, we found no consistent candidate–control difference. Candidates do, however, show greater causal strength and a higher threshold-testable fraction than randomly sampled same-locus controls (Appendix D, Figure 10).

Q3

Yes. We do not need a matched clean model, provided we have access to a sample of benign/attack prompts and known trigger information.

During rebuttal, we conducted an additional poisoning-defense experiment using BackdoorLLM’s independently trained Llama-2-7B BadNets LoRA adapter [2]. We applied CHA without a matched clean model. This extends our Grammar evaluation to another (larger) model, poisoning attack, and safety-related, open-ended generation task. Using 50+ BackdoorLLM attack/benign prompts, we identified 20 candidate channels; 17 showed positive defense leverage (+10.0 to +32.1 pp; i.e., attack suppression minus benign damage). To test transfer to different prompts, we selected three highest-leverage channels and evaluated them on 100+ HarmBench harmful/benign prompts. Defense leverage was +21.0 (m1:3135), +19.0 (m1:2469), and +12.9 (m1:2350). Notably, each intervention replaces a single channel.

Regarding generalization across training checkpoints, our original submission investigates this question. Across three Grammar poisoning runs, current-checkpoint selection achieves median singleton defense leverage of +6.2, +44.9, +36.5, and +23.5 pp at 25%, 50%, 75%, and 100% of training, respectively. In contrast, targets selected at previous checkpoints exhibit negative median defense leverage (Fig. 6 and Appendix Table 9). Together with RQ1–RQ3, these results show that high causal reach alone does not make a fixed intervention reliable. Targets should therefore be reselected and validated at each checkpoint being defended (which is practical though, as explained in our responses to Reviewer honh).

Finally, we emphasize that the matched clean reference is an assumption of our controlled fine-tuning setting, not an intrinsic requirement of CHA. Similar reference-based assumptions appear also in prior work [3,4].

References

  1. Fourcaud-Trocmé et al. (2003). How spike generation mechanisms determine the neuronal response to fluctuating inputs.
  2. Li et al. (NeurIPS 2025). BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models.
  3. Li et al. (EMNLP 2024). CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models.
  4. Zeng et al. (EMNLP 2026). Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoders.

Thank you for the rebuttal. My two main concerns remain, so I keep my score.

The spiking-like threshold is the paper's central claim, and the evidence still does not separate it from an ordinary decision-boundary effect. The response describes the term as a descriptive analogy, and the paper itself notes that a smooth latent trajectory can cross a decision boundary in this way (Section 7). A single persistent crossing is also expected when one channel is interpolated linearly, and the peak margin change at the crossing is only 4.8 pp more frequent than at adjacent intervals. The answer to Q2 points the same way. Matched controls with comparable causal strength show the same threshold behavior, so the threshold seems to follow causal strength and is not specific to overtopping channels.

The new experiment drops the clean reference model but relies on known trigger information and attack prompts. This is conflicts the principle set earlier. Section 6 states that channel selection does not use trigger annotations or attack outcomes. It also does not yet appear in the revised PDF. On later checkpoints, the response confirms that previously selected targets have negative defense leverage, and the paper notes that same-checkpoint selection does not establish prospective transfer. The defense works only when targets are reselected and evaluated at the same checkpoint, and it seems not to hold up as training continues.

We want to clarify several inaccuracies in Reviewer Un4a's response and interpretation of our results, particularly since this discussion will remain public.

  1. Spiking-like behaviour. The reviewer states that "the evidence still does not separate it from an ordinary decision-boundary effect." However, the paper explicitly goes beyond binary decision-boundary crossings, as shown by the continuous logit-margin and temporal analyses in Section 5 (Fig. 4a–c), further clarified in our rebuttal. The reviewer also argues that "threshold-like changes in binary outputs can arise when smoothly changing internal representations cross a decision boundary." This repeated framing of smooth nonlinear decision-boundary effects as an alternative to spiking-like behaviour is misleading: even spike initiation in the EIF model arises from smooth nonlinear dynamics. Our claim concerns spiking-like response geometry, not a distinct internal spiking mechanism. Moreover, dismissing the 4.8 pp result as "only 4.8 pp" overlooks that this measures excess event localization frequency, not the magnitude of the causal effect. The result is statistically significant (p < 0.0001) and is complemented by additional statistically significant evidence of margin-change concentration and temporal localization (Fig. 4b–c).

  2. Matched controls. The reviewer concludes that threshold behaviour "is not specific to overtopping channels" because "matched controls with comparable causal strength show the same threshold behavior." However, our answer to Q2 explicitly states that controls with comparable high singleton causal strength are, by definition, also overtopping channels, regardless of how they were discovered. Therefore, the reviewer's conclusion does not follow logically from the comparison: the comparison does not establish threshold behaviour outside overtopping, only outside CHA-discovered candidates.

  3. Poisoning experiments. These experiments are explicitly presented as a proof of concept. The reviewer states that the additional experiment using known trigger information "conflicts the principle set earlier." However, this experiment was specifically conducted to remove the clean-reference requirement in response to the reviewer's question, under different experimental assumptions. It does not invalidate the original experiments. Furthermore, the reviewer argues that "the defense works only when targets are reselected and evaluated at the same checkpoint" and "seems not to hold up as training continues." This is precisely one of the paper's main findings: static interventions become unreliable as training continues. We therefore do not see how restating this result contradicts the paper's claims.

Finally, we have not uploaded a revised PDF. All additional experiments and clarifications were provided in the rebuttal. The reference to a revised PDF is therefore incorrect, as no such version exists.

Thank you for the detailed clarifications.

They resolve several of my earlier points, in particular the meaning of the 4.8 pp result and the status of the matched controls. However, the paper still does not show what the term "spiking-like" adds beyond a sharp nonlinearity near the decision boundary, since Figure 4 uses equal-change and adjacent-interval references but no smooth nonlinear baseline.

The practical implications of the defense experiments also remain limited, since the new setting assumes known triggers and attack prompts, and the paper presents the defense as a proof of concept without prospective transfer.

The BackdoorLLM experiment in the response may remove the need for a matched clean model, but it is not in the current paper because no revised PDF has been uploaded, so I cannot credit it and take it into account in my assessment.

Reviewer_f5s2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates overtopping, a phenomenon in which intervening on a single activation channel can significantly affect model behavior across many examples. The authors systematically study this phenomenon from four perspectives: its relationship with task competence, interactions among influential channels, sensitivity to intervention strength and timing, and stability throughout training. The experiments reveal that high-leverage channels often exhibit spiking-like behavior, where model outputs remain stable until the intervention strength crosses a certain threshold. The authors also find that jointly intervening on multiple influential channels can suppress their individual effects, and that causal influence can shift between channels during training even when overall task performance remains stable. Finally, through a controlled backdoor poisoning experiment, the paper demonstrates that static intervention targets can become ineffective as training progresses, while checkpoint-specific interventions can provide more effective suppression of unwanted behavior.

Critical strengths
  1. Comprehensive experimental analysis. The authors investigate overtopping across five tasks and multiple model families, considering intervention magnitude, generation timing, channel interactions, and training dynamics. The use of held-out evaluations and structurally matched control channels strengthens the experimental findings.

  2. Interesting findings on intervention dynamics. The observed threshold-like behavior and non-additive interactions among channels provide useful insights into how localized causal control operates in LLMs. The backdoor poisoning experiment further demonstrates the potential practical relevance of these findings.

Critical weaknesses
  1. The central claim is that influential channels exhibit spiking-like causal behavior. However, the observed threshold transitions may simply result from continuous changes in logits crossing a discrete decision boundary. Although the authors analyze logit-margin changes, the paper does not sufficiently establish that the underlying computation exhibits a distinct threshold mechanism rather than ordinary nonlinear decision behavior. Stronger comparisons against smooth-response baselines would help justify the spiking analogy.
  2. The paper primarily extends the previously introduced overtopping phenomenon through additional empirical analyses. While the observations are interesting, the work does not introduce a fundamentally new intervention algorithm or provide a theoretical explanation of why these patterns emerge. The distinction between the proposed characterization and existing findings on activation steering, causal mediation, and nonlinear intervention effects could be clarified.
  3. Most experiments use relatively small language models, with the largest model containing 7B parameters. Additionally, the model, task, and intervention settings are not fully crossed. Since overtopping depends on the activation basis, intervention method, and task definition, it is unclear how broadly the reported patterns generalize to larger models, different representations, or more open-ended generation tasks.
Questions for authors
  1. Since the behavioral outcomes are binary, a smoothly changing logit margin could naturally produce an abrupt output transition. Could the authors provide stronger evidence that overtopping reflects a distinct threshold-like computational mechanism?
  2. Several observations, including nonlinear intervention effects and changing causal representations during training, are related to existing mechanistic interpretability findings. Could the authors clarify which conclusions are specific to overtopping and which represent more general properties of activation interventions?
  3. Could the authors evaluate the proposed intervention strategy on larger language models, more diverse backdoor attacks, or open-ended generation tasks, and compare it against existing activation-based defense methods?
1 comment

Q1

We agree that the behavioral evaluation is binary, although the model generates nonbinary answers or full continuations (e.g., BoN jailbreaks); we score correctness, validity, or safety (Section 2 and Appendix A).

Smooth logit changes can produce abrupt output transitions, and this is compatible with our spiking-like analogy. In the EIF model, spike initiation itself arises from smooth nonlinear dynamics.

RQ3 goes beyond binary transitions. Overtopping channels are identified through their behavioral effects, so observing a sudden output change is not what motivates our use of spiking-like. We examine instead how the behavioral effects develop. Specifically, we find that the largest changes in logit margins tend to occur around behavioral transitions (Fig. 4a), that stronger channels concentrate more of their total absolute margin change into one interval (Fig. 4b), and that causal sensitivity is often concentrated at particular generation steps (Fig. 4c). These continuous-margin and temporal response properties motivate our use of spiking-like: they go beyond discrete output changes and are not trivial or guaranteed by construction.

Our interpretation, formalized in Appendix D, is that the behavior of overtopping channels resembles that of the EIF model along the dimensions discussed above, without implying that LLMs implement EIF dynamics or spiking. This analogy motivates our investigation; our contribution is to formulate testable response properties and provide experimental evidence. Whether these observations reflect a shared underlying computational principle or arise through different mechanisms remains an important question for future research. We will make this distinction even clearer in our paper.

Q2

Previous work demonstrated activation steering [2], interactions between causal mediators [3], changes during learning [4], and overtopping [1]. These general phenomena are not claimed to be unique to overtopping. To our knowledge, we provide the first joint characterization of the reliability of individual overtopping channels as intervention targets.

Specifically, we jointly test how singleton, high-reach overtopping channels behave when combined, when intervention strength or decoding time varies, and after further training. The poisoning experiments illustrate why fixed targets can fail and checkpoint-aligned targets may help, offering a direction for targeted backdoor mitigation.

Q3

We agree that broadening the evaluation would strengthen our findings. To address this, we conducted two additional experiments during rebuttal to extend our evaluation across model scales (1B–72B), additional backdoor attacks, and open-ended generation tasks:

  1. Larger models: We evaluated 4-bit NF4 Qwen2-72B-Instruct on arithmetic (accuracy: 0.914). A full search identified 7 overtopping channels, with singleton-union reach of 0.306 (0→1) and 1.0 (1→0). The strongest singleton reached 0.993 (1→0). The smaller number of overtopping channels at higher accuracy is consistent with observations in RQ4. Matched 7B NF4 experiments also showed overtopping (77 channels).
  2. Additional backdoor attack on larger models and open-ended generation: To test beyond Grammar, we evaluated BackdoorLLM's Llama-2-7B BadNets [5], which induces harmful generation, using its original prompt and decoding settings. Also in this experiment, overtopping channels showed positive defense leverage (+10 to +32.1 percentage points). To test transfer to different prompts, we selected three highest-leverage channels and evaluated them on 100+ HarmBench harmful/benign prompts. The resulting defense leverage was +21.0 for m1:3135, +19.0 for m1:2469, and +12.9 for m1:2350.

Finally, our primary goal is to characterize overtopping's causal properties, not to develop a competitive backdoor defense. The poisoning experiments serve as a proof of concept, showing that intervening on a single channel can suppress unwanted behavior with limited damage. We do not claim superiority over existing activation-based defenses (a comparison we leave to future work); our contribution is to demonstrate that meaningful defense leverage can be achieved through such a minimal intervention.

References

  1. Sovrano et al. (KDD 2026). Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation. https://dl.acm.org/doi/abs/10.1145/3770855.3818091
  2. Rimsky et al. (ACL 2024). Steering Llama 2 via Contrastive Activation Addition. https://aclanthology.org/2024.acl-long.828/
  3. Vig et al. (2020). Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias. https://arxiv.org/abs/2004.12265
  4. Nanda et al. (ICLR 2023). Progress Measures for Grokking via Mechanistic Interpretability. https://arxiv.org/abs/2301.05217
  5. Li et al. (NeurIPS 2025). BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models. https://github.com/bboylyg/BackdoorLLM
Reviewer_honhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks at how changing one small part of an LLM can sometimes change its behavior across many questions. The authors find that these changes often happen suddenly, and that the important parts of the model can change as it learns. They also show that targeting these parts could reduce harmful behaviors, but because they change, the targets need to be updated as the model continue training.

Critical strengths
  • in addition to identifying which parts of the model are important, they also look at how and when they affect its behavior
  • tests several methods and tasks
  • tests how much intervention is needed and when during generation it matters most
  • interesting finding that individually effective change can interfere with each other
Critical weaknesses
  • harmful behavior experiment is daily simple, not sure whether the approach would work for more complicated attacks
  • methods requires comparing the model with a clean version, which might not always be available in real life
  • I’m not fully convinced by the comparison to spiking neurons since the sudden changes might just happen because the model crosses a decision boundary (Section 5, Section 7)
Questions for authors
  • since the important components keep changing during training, could we find patterns that help predict where it will move to next?
  • how practical would it be to repeatedly search for these components in much larger models? if we were to use this as a real defense
  • could this approach work if we don’t have a clean model to compare against?
1 comment

Q1

That's an interesting direction, but our results suggest that predicting where causal leverage will move during training is quite challenging. As shown in RQ4 (Section 6, Figures 5–6), causal leverage can reorganize substantially across training checkpoints, even when ordinary task performance remains relatively stable. Previously influential channels can also lose their causal effect entirely or even become counterproductive when reused at later checkpoints.

In particular, Figure 5 shows that overtopping reach can change non-monotonically during training, while Figure 6 and Appendix Table 9 demonstrate that interventions selected at previous checkpoints do not reliably transfer to subsequent ones. This makes it difficult to assume that causal importance follows a smooth or predictable trajectory. With our current methods, we can identify influential channels at a given checkpoint, but we cannot reliably predict which channels will become important next, or whether previously important channels will lose their influence.

Identifying patterns in this reorganization would therefore be an interesting future direction, as also indicated in Section 9 (Conclusion), where we suggest investigating whether circuit topology, redundancy, response curvature, or training dynamics could help predict where causal access moves.

Q2

The underlying search procedure for overtopping channels, i.e., CHA [1], has an asymptotic time complexity of O(L [k log(N/k) + k]) group interventions for L layers, N channels per layer, and k above-threshold channels per layer. Therefore, the search is logarithmic in N for fixed L and k (in practice, in our experiments we observe that k is often tiny, i.e., the 95th percentile is below 30).

Thus, thanks to the logarithmic search, under the assumptions underlying CHA [1], finding a (not necessarily complete) set of overtopping channels in much larger LLMs is computationally feasible. To show this in practice, we have also extended our evaluation to Qwen2-72B-Instruct, successfully running a complete CHA search in a matter of hours and confirming both the presence of overtopping and the feasibility of CHA-based channel discovery at the 72B scale.

Q3

Yes. We do not need a matched clean model, provided we have benign examples and labeled attack examples with known trigger information. To demonstrate this, we conducted an additional poisoning-defense experiment during rebuttal using BackdoorLLM [2], a benchmark providing independently trained backdoored models. Unlike our Grammar experiments (Section 6), this evaluation considers a more challenging, safety-related task involving substantially longer inputs and generated outputs. Specifically, we applied CHA to BackdoorLLM’s released Llama-2-7B BadNets LoRA adapter [2], using known trigger information and benign questions, without relying on a matched clean model.

Using 50+ BackdoorLLM attack/benign prompts, we identified 20 candidate channels, 17 of which exhibited positive defense leverage (+10.0 to +32.1 percentage points; i.e., attack suppression minus benign damage). To test transfer to different prompts, we selected three highest-leverage channels and evaluated them on 100+ HarmBench harmful/benign prompts. The resulting defense leverage was +21.0 for m1:3135, +19.0 for m1:2469, and +12.9 for m1:2350. Notably, each intervention replaces a single coordinate.

Importantly, the underlying overtopping-channel search procedure [1] itself does not require a clean reference model. The clean model used in our Grammar experiments (Section 6) was required only for selecting channels without using trigger information or attack outcomes. We consider this a reasonable assumption for controlled training and fine-tuning auditing, rather than a universal requirement of our approach. Similar reference-based assumptions appear also in prior work [3,4]. We agree that such access may not be available when only the final model is provided. We will explicitly clarify it in the revised manuscript.

References

  1. Sovrano et al. (KDD 2026). Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation. https://dl.acm.org/doi/abs/10.1145/3770855.3818091
  2. Li et al. (NeurIPS 2025). BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models. https://github.com/bboylyg/BackdoorLLM
  3. Li et al. (EMNLP 2024). CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models. https://doi.org/10.18653/v1/2024.emnlp-main.514
  4. Zeng et al. (EMNLP 2026). Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoders. https://doi.org/10.18653/2026.findings-emnlp.198

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Since we are approaching the end of the rebuttal phase, for your convenience, we would like to briefly summarize how we addressed the main questions and concerns raised during the discussion:

  1. Spiking-like interpretation and threshold behaviour. We clarified that spiking-like refers to measured causal response properties, rather than an internal spiking mechanism. We explained how our continuous logit-margin and temporal analyses go beyond binary decision-boundary crossings. We also clarified that the matched-control results do not establish that threshold behaviour is unrelated to overtopping, since controls with comparable high singleton causal reach are themselves overtopping channels. The comparison only shows that these properties are not exclusive to CHA-discovered candidates.

  2. Novelty and scope of the contribution. We emphasized that our contribution is the joint characterization of high-leverage singleton channels (i.e., overtopping channels) as intervention targets, examining how their causal effects depend on intervention strength, decoding time, interactions with other channels, and further training. Together, these properties help explain why static interventions can become unreliable, even when individual channels initially exhibit high causal leverage.

  3. Scalability and generalization to larger models. We extended our evaluation to Qwen2-72B-Instruct, confirming the presence of overtopping at the 72B scale. We also demonstrated the practical feasibility of CHA-based discovery.

  4. Generalization to more realistic poisoning settings and clean-reference requirements. We conducted an additional experiment using BackdoorLLM's Llama-2-7B BadNets adapter, extending our evaluation to a safety-related, open-ended generation task without requiring a matched clean model. We identified 20 candidate channels, 17 with positive defense leverage (+10.0 to +32.1 pp). We further evaluated three channels on 100+ HarmBench prompts, obtaining positive defense leverage (+12.9 to +21.0 pp). We also clarified the different assumptions underlying the original and additional poisoning experiments.

  5. Stability and predictability of intervention targets during training. We reiterated that our original experiments already demonstrate how previously effective intervention targets can lose their causal leverage or become counterproductive after further training. While current-checkpoint measurements can identify useful targets, these results show why fixed interventions cannot be assumed to remain effective as training continues. We also discussed why predicting the future reorganization of causal leverage remains an open research question.

Unfortunately, most reviewers did not participate in the rebuttal discussion phase, leaving limited opportunity for further feedback on our responses. Nevertheless, we hope that the additional experiments and clarifications provided in our rebuttal will assist the meta-reviewer in evaluating how the reviewers' main concerns have been addressed, as well as the scope and implications of our findings.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.