acceptodds
Withdrawn from ICLR 2027

When Attribution Graphs Fail to Predict LLM Jailbreaks: Distinguishing Structural Signals, Causal Effects, and Behavioral Safety

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Internal attribution graphs offer a promising way to examine how large language models process adversarial prompts, but their ability to predict, explain, and mitigate jailbreak behavior remains unclear. We investigate these three objectives separately using full-depth, transcoder-based MLP-feature attribution graphs for Llama-2-7B-chat. Across 50 topically adjacent clean–adversarial prompt pairs, we align graphs by exact feature identity and evaluate four structural measures: graph deviation, feature suppression, feature emergence, and path rerouting. None reliably distinguishes successful from failed jailbreaks under the evaluated setting, and the binary outcomes themselves are sensitive to decoding: only two of five successes observed with sampled decoding persist under greedy decoding. We next rank features by gradient-based sensitivity and screen them through targeted interventions. Ablating the selected features produces substantially larger changes in target-token loss than matched random-feature interventions, demonstrating that locally influential internal components can be identified even when aggregate graph statistics are not predictive. However, follow-up evaluation of complete generations shows that satisfying the loss-based intervention criterion does not reliably restore refusal behavior. These results reveal a three-way dissociation between structural prediction, causal sensitivity, and behavioral mitigation. Internal attribution graphs, as operationalized here, can localize computation affecting a target prediction but do not yet provide reliable jailbreak detectors or behaviorally validated defenses. Our findings highlight the need for decoding-stable outcome definitions, behavior-level intervention evaluation, and independently validated structural signatures in mechanistic studies of language-model safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large language models, LLM jailbreaks, mechanistic interpretability, attribution graphs, sparse transcoders, causal interventions, behavioral safety, jailbreak mitigation, adversarial robustness, AI safety.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hjHLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether internal attribution graphs can reliably predict jailbreak success, identify causally influential model components, and support behavioral safety interventions. The authors construct full-depth, transcoder-based attribution graphs for Llama-2-7B-chat and analyze 50 clean–adversarial prompt pairs using four structural metrics: graph deviation, safety suppression, attack emergence, and path rerouting.

The experimental results show that none of these structural metrics exhibits a statistically significant association with jailbreak success. However, only five of the 50 adversarial prompts are classified as successful, and just two of these successes persist under greedy decoding, revealing substantial sensitivity to the evaluation procedure.

The authors further investigate causal interventions by selecting internal features through gradient-based sensitivity analysis. Ablating these features produces substantially larger changes in target-token loss than random-feature interventions. Nevertheless, these token-level effects do not reliably translate into restored refusal behavior in complete generations.

The paper concludes that structural prediction, causal sensitivity, and behavioral safety should be treated as distinct objectives. While attribution graphs may help localize components influencing individual predictions, the evaluated structural metrics do not provide reliable jailbreak prediction, and successful token-level interventions do not necessarily establish effective behavioral mitigation.

Critical strengths
  1. Important and relevant research question.

The paper addresses an important question in mechanistic interpretability and LLM safety: whether internal attribution graphs can provide reliable indicators of jailbreak vulnerability and support effective safety interventions. Distinguishing structural prediction, causal sensitivity, and behavioral mitigation is conceptually meaningful, particularly given the growing interest in using internal model representations for safety assessment.

  1. Useful empirical observations regarding the limitations of attribution-based analysis.

The paper provides interesting negative findings showing that four structural metrics—graph deviation, safety suppression, attack emergence, and path rerouting—do not exhibit statistically significant associations with jailbreak success in the evaluated setting. These observations highlight potential limitations of relying on aggregate structural measurements as indicators of adversarial vulnerability.

  1. Incorporation of causal interventions and control experiments.

Beyond examining structural correlations, the authors investigate whether selected internal features causally influence model predictions. The comparison against random-feature interventions provides evidence that sensitivity-ranked features can produce meaningful changes in target-token loss. This adds an experimental dimension beyond purely descriptive graph analysis.

  1. Recognition of important evaluation limitations.

The paper explicitly acknowledges that token-level intervention effects do not necessarily translate into restored refusal behavior. It also reports the sensitivity of jailbreak success labels to decoding strategies and includes additional statistical analyses. This transparency is valuable for interpreting the findings and identifying methodological challenges in mechanistic safety evaluation.

Critical weaknesses
  1. Insufficient experimental evidence to support the central negative finding.

My primary concern is the limited statistical evidence supporting the paper's central conclusion. Among the 50 adversarial prompts, only five are classified as successful jailbreaks. This highly imbalanced and small sample makes it difficult to reliably assess the relationship between structural metrics and jailbreak success. Although the authors report nonsignificant correlations and additional statistical tests, the absence of statistical significance does not establish the absence of predictive relationships. Several reported confidence intervals remain relatively wide, leaving meaningful associations plausible. Consequently, the experiments support the narrower observation that no reliable association was detected in this setting, rather than a general conclusion that attribution-graph metrics are ineffective for jailbreak prediction.

  1. Limited generalizability due to the restricted experimental setting.

The evaluation is conducted primarily on a single model, Llama-2-7B-chat, using only 50 clean–adversarial prompt pairs. The results are not validated across different model families, model sizes, alignment strategies, or a sufficiently diverse collection of jailbreak attacks. Given that internal representations and safety mechanisms can differ substantially across models, it remains unclear whether the observed negative findings reflect a general limitation of attribution-based safety analysis or characteristics of the specific model and attack setting.

  1. Incomplete graph representation limits the interpretation of the results.

Although the paper introduces a relatively general attribution-graph framework, the experimental implementation primarily relies on MLP transcoder features and does not explicitly instantiate attention, embedding, or residual components as graph nodes. These omitted components may contribute to safety-relevant computations and adversarial behavior. Therefore, the failure of the evaluated structural metrics cannot be readily generalized to attribution graphs more broadly. It remains unclear whether the negative results arise from fundamental limitations of structural attribution or from the particular graph representation and aggregation metrics employed.

  1. Insufficient behavioral validation of the causal intervention results.

The paper demonstrates that interventions on sensitivity-ranked features can substantially change target-token loss compared with random-feature controls. However, these effects do not establish successful behavioral mitigation. More importantly, complete post-intervention generations were not systematically evaluated for all successful attacks in the primary experiment, and the decoding-matched behavioral follow-up is limited. A more comprehensive evaluation using repeated generations, consistent decoding settings, and explicit refusal or harmfulness measurements is necessary to substantiate the claimed distinction between token-level causal effects and behavioral safety.

  1. Limited methodological novelty and insufficient isolation of failure mechanisms.

The proposed analysis largely combines existing techniques, including structural graph comparisons, feature activation measurements, gradient-based sensitivity analysis, and activation interventions. While their application to jailbreak analysis is relevant, the paper does not clearly establish a substantial methodological advance over existing mechanistic interpretability approaches. Furthermore, the experiments do not adequately isolate whether the observed predictive failures originate from the graph construction procedure, feature alignment, the selected aggregate metrics, or instability in jailbreak success labels. Without disentangling these factors, the study provides limited insight into how the identified limitations could be addressed.

  1. Limited practical significance of the findings in their current form.

The distinction between structural correlation, token-level causal influence, and behavioral safety is important. However, the current experiments do not establish sufficiently generalizable conditions under which attribution-based methods succeed or fail, nor do they provide a validated alternative for improving jailbreak detection or mitigation. As a result, the broader implications for designing reliable mechanistic safety methods remain uncertain.

Questions for authors

Major Questions

  1. Statistical validity of the central negative finding

The paper's central conclusion relies on the observation that none of the four structural metrics significantly predicts jailbreak success. However, only five of the 50 adversarial prompts are classified as successful jailbreaks. Given this small and highly imbalanced sample, how can the authors distinguish a genuine lack of predictive signal from insufficient statistical power? In particular, several confidence intervals appear wide enough to accommodate potentially meaningful associations. Could the authors provide additional experiments with a substantially larger number of successful jailbreaks to strengthen this conclusion?

  1. Generalizability across models and attack strategies

The main experiments are conducted on Llama-2-7B-chat using a relatively limited set of adversarial prompts. To what extent do the reported findings generalize to other model families, larger or more recent instruction-tuned models, and stronger jailbreak strategies? Could the authors evaluate whether the same structural metrics fail to predict jailbreak success across different model architectures and alignment mechanisms? Without such validation, it is difficult to determine whether the findings reflect a general limitation of attribution-based analysis or a model-specific phenomenon.

  1. Completeness of the attribution graph representation

Although the proposed framework describes a general attribution graph, the experimental implementation primarily uses MLP transcoder features without explicitly representing attention, embedding, or residual components as graph nodes. How might these omitted components affect the structural metrics and their relationship with jailbreak success? Could the authors provide evidence that the observed negative results are not primarily caused by the restricted graph representation? More broadly, what justifies extending the conclusions beyond the particular graph construction method evaluated in this study?

  1. Behavioral validation of causal interventions

The paper shows that interventions on sensitivity-ranked features produce larger changes in target-token loss than random-feature controls. However, the primary experiments do not systematically evaluate complete post-intervention generations for all successful attacks, and the behavioral follow-up is limited. Could the authors provide a more comprehensive evaluation of actual refusal restoration and harmful-response reduction, using repeated generations and consistent decoding settings? How often do interventions that satisfy the token-level loss criterion lead to meaningful improvements in behavioral safety?

  1. Methodological novelty relative to existing interpretability approaches

The proposed framework combines structural graph comparisons, feature activation analysis, gradient-based sensitivity ranking, and causal interventions, all of which have substantial connections to existing mechanistic interpretability methods. What is the main methodological contribution beyond applying these established techniques to jailbreak analysis? Could the authors clarify which components are genuinely novel and compare the proposed analysis against simpler existing attribution or activation-based approaches under matched experimental conditions?

  1. Identifying the actual causes of structural prediction failure

The paper attributes limited predictive performance to the evaluated structural metrics, but several alternative explanations remain possible, including feature-alignment limitations, coarse graph aggregation, incomplete graph construction, and unstable jailbreak success labels. Can the authors experimentally disentangle these factors? For example, would more localized, safety-specific graph features or alternative attribution measures yield stronger predictive relationships? Without such analysis, how can the authors determine whether the observed failure is intrinsic to structural attribution or specific to the chosen metrics?

Minor Questions

  1. Stability of jailbreak success labels

Only two of the five sampled-decoding jailbreak successes persist under greedy decoding. Have the authors evaluated attack success rates across multiple random seeds and repeated sampled generations? How sensitive are the reported correlations to alternative definitions of jailbreak success, such as estimated success probabilities rather than binary single-generation labels?

  1. Sensitivity to structural metric design

The paper evaluates four structural metrics, some of which depend on feature activation thresholds or aggregate node counts. Although threshold sensitivity is examined, have the authors considered normalized metrics that explicitly control for graph size, prompt length, or the number of active features? Could these factors obscure safety-relevant structural differences?

  1. Strength of the causal intervention baselines

The comparison with random-feature interventions demonstrates that sensitivity-ranked features influence the selected token-level objective. However, does the proposed selection strategy outperform simpler alternatives, such as activation-magnitude ranking or direct gradient-based feature selection without attribution-graph construction, when evaluated under identical intervention budgets? This comparison would help establish whether the graph-based framework provides additional value.

  1. Practical implications and future applicability

Given that the evaluated structural metrics do not reliably predict jailbreak success and the interventions do not consistently restore refusal behavior, what concrete practical advantages does the proposed framework currently offer over existing behavioral jailbreak evaluation methods? What evidence would be required to establish that attribution graphs can become useful components of a reliable safety assessment or mitigation pipeline?

Reviewer_19shOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper asks whether attribution graphs can predict and prevent jailbreaks. In Llama-2-7B-chat experiments, four graph statistics fail to distinguish successful attacks from failures, and the results also depend on decoding. Selected features can strongly affect next-token loss, but ablating them does not reliably make the model refuse. Attribution graphs can therefore locate output-relevant computation, but are not yet reliable jailbreak detectors or defenses.

Critical strengths

None

Critical weaknesses
  • The central claim is broader than the evidence. The paper claims that attribution graphs fail to predict jailbreaks, but it tests only four aggregate statistics on one model and 50 custom prompt pairs, with only five positive cases. The analysis reports in-sample correlations rather than held-out predictive performance, and only two of the five positive labels remain under greedy decoding.
  • The paper lacks a systematic discussion and empirical comparison of directly related work. It omits major safety-intervention methods such as Circuit Breakers [1] and TraceRouter [2]. A direct conceptual and experimental comparison with them is necessary to justify the contribution and the conclusion of this paper.

[1] Zou, A., Phan, L., Wang, J., Duenas, D., Lin, M., Andriushchenko, M., ... & Hendrycks, D. (2024). Improving alignment and robustness with circuit breakers. Advances in Neural Information Processing Systems, 37, 83345-83373.

[2] Shi, C., Li, S., Lu, W., Wu, W., Wang, C., Cheng, Z., ... & Chua, T. S. (2026). TraceRouter: robust safety for large foundation models via path-level intervention. arXiv preprint arXiv:2601.21900.

Questions for authors

Q1: Can you increase the number of prompt pairs and successful attacks and evaluate additional models? With only 50 pairs, five positives and one model, it is unclear whether the negative conclusion generalizes beyond this setting.

Q2: Can you explain why the main conclusion of this paper conflicts with existing work such as Circuit Breakers and TraceRouter?

Reviewer_MpdAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether transcoder-based attribution graphs can predict, explain, or mitigate jailbreaks on Llama-2-7B-chat. The authors train per-layer MLP transcoders (4,096 features) and build full-depth graphs for 50 clean/adversarial prompt pairs. Clean and attacked graphs are aligned by exact (layer, feature, position) identity, and four aggregate metrics are computed: graph deviation, suppression, emergence and path rerouting. None correlates with jailbreak success (5 successes out of 50; only 2 of 5 persist under greedy decoding). Ablating features ranked by gradient sensitivity changes target-token loss far more than random features (53–987× on 2 pairs), but the ablated model still complies in the two pairs where full generations were inspected. The authors conclude there is a “three-way dissociation” between structural prediction, causal sensitivity and behavioral safety.

Critical strengths
  1. The authors are candid, I liked their honesty. They separate prediction, causal effect and behavior, report that an earlier significant result did not replicate (r=0.461 on 30 pairs vs. −0.218 on 50), and report that labels depend on decoding. They also use bootstrap intervals, Benjamini–Hochberg correction and a threshold sweep. The messages that loss-based intervention criteria are not behavioral evidence, and that jailbreak labels need repeated sampling, are worth stating in the interpretability-for-safety literature.

  2. The effort is real: full-depth graphs on a 7B model with error nodes, and an honest account of how hard transcoder training was.

Critical weaknesses
  1. The negative result is not informative, and the generalization in the title is not thoroughly supported. By design, edges are position-locked and attention is not modeled, so no cross-token pathway can appear in the graph. Graphs are unpruned (137k–355k nodes) and nodes are keyed by absolute position, so deviation, suppression and emergence mostly count extra tokens in longer attack prompts. Node counts are almost identical across pairs, and the metrics have small variance. Path rerouting is bimodal (near 0 or 6), dominated by a few outlier edges (Fig. 2). The self-trained transcoders are weak where it matters (FVE 0.21 at layer 3, 0.44–0.70 in layers 2–12). Without error nodes, KL is 32.6 vs. 0.0016, so most computation sits in error nodes. Statistically, there are 5 positives with labels that flip under greedy decoding. The keyword rule disagrees with a judge on 28% of cases, and the 50 pairs come from 25 templates. The confidence intervals still include sizable effects (suppression up to +0.48, rerouting down to −0.57), and the power calculation only covers r=0.4. Everything is one old model, custom prompts and one transcoder recipe. Attribution graphs are meant to generate hypotheses to test with interventions, not to serve as predictors. There is no comparison with cheap baselines (prompt length, refusal-direction probes).

  2. The causal and behavioral legs are tautological or rest on n=2. Features are chosen by first-order sensitivity of the target loss, and success is |ΔL|>0.05 or a top-1 change, in either direction, so 5/5 is nearly guaranteed. The 53–987× advantage over random features, which the abstract states generally, is from 2 pairs, and it is not stated that the random features are active. Full generations were inspected on those 2 pairs only, and both real and random ablations still complied. Yet Section C.3 calls a single newline output “successful mitigation”. The “dissociation” is therefore an anecdote on the behavioral side. Novelty is also unclear. The framing that earlier attribution methods are single-input is inaccurate (attribution patching and circuit tracing are paired or intervention-based), and the sections on jailbreak interpretability and transcoders omit that work. Section D.3 claims the work “demonstrates” that transcoder attribution “detects jailbreak mechanisms” and offers a “scalable framework”, contradicting the results and D.4 (the alignment cannot generalize across models and the motif search does not scale). Appendices E–J mostly document superseded iterations and hand-picked examples. Practical impact is limited, since a 32-layer graph per prompt is far costlier than a probe.

Questions for authors
  1. Can you re-run with token-aligned or length-normalized alignment, graphs pruned to paths that reach the target logit, and attention and cross-position edges? Do any metrics change?

  2. Can you define success as a rate over k≥20 samples per prompt with a validated judge, use it as a continuous target, and enlarge the sample from standard benchmarks and attacks (AdvBench, JailbreakBench, GCG/PAIR)? What is the power for r≈0.2–0.3 under your label noise, and can you use cluster-robust inference over the 25 templates?

  3. For every successful pair, can you report full generations after ablation with a validated refusal judge, plus activation-matched random controls and the magnitude baseline?

  4. Why train weak transcoders for Llama-2 rather than use better-fidelity public ones, and does the result hold with them or on a second model with the full pipeline?

  5. Can you reconcile Section D.3 (“detects jailbreak mechanisms”, “scalable”) with the results, and compare against a simple probe or prompt-length baseline for “prediction”?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.