acceptodds
Under review as a conference paper at ICLR 2027

Eliciting sandbagged LLM capability via structural interventions on attention-head functional networks

Abstract

Reliably evaluating the capabilities of advanced AI systems is critical for safe development and deployment, yet such evaluations may be undermined by sandbagging, in which a model strategically suppresses capabilities it possesses. Existing work has developed controlled sandbagging settings and methods for detecting or eliciting hidden capabilities, but comparatively little is known about how a model’s internal computation changes when its capabilities are suppressed. In this work, we study sandbagging through the lens of internal functional organization. Inspired by network neuroscience, we construct functional graphs from attention-head activations and compare their organization under honest and sandbagging conditions while holding the model architecture and weights fixed. Across model architectures and community detection methods, we find a consistent structural signature: sandbagging is associated with reduced functional connectivity within detected communities. We further show that this reorganization provides an actionable signal for capability elicitation. Ablating attention heads that exhibit the largest connectivity changes substantially recovers suppressed performance while largely preserving performance under honest behavior. Moreover, incorporating these structural signals further improves existing inference-level and weight-level elicitation methods. Together, our results show that sandbagging is accompanied by systematic reorganization of internal computation and demonstrate that functional network structure can provide both a mechanistic lens for understanding capability suppression and a practical signal for recovering hidden capabilities.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Sandbagging, Functional Networks of Attention Heads

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LfZvOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript considers the problem of eliciting sandbagged capabilities of large language models. Here, sandbagging refers to the phenomenon that at evaluation time, models suppress part of their capabilities, resulting in a lower-than-actual evaluation performance. The authors approach this problem with a mechanistic interpretability method. At the high level, they first propose the concept of a "functional network", which is an undirected graph, where the vertices are different attention heads, and the edges consist of the top 25% pairs of attention heads with the most activation similarity under a training set. They construct such functional network for normal LLMs and LLMs performing controlled sandbagging. Additionally, they construct a third graph, on which they cluster vertices to obtain "communities" of attention heads. The authors identified an empirical correlation between sandbagging behavior, and reduced connectivity within such communities in the functional network. They use this finding to propose interventions to recover sandbagging LLM's capabilities, including ablating heads that significantly changes connectivity when instructed to sandbag, and fine-tuning weights on such heads. Empirically, they are able to show that the proposed interventions are effective on Mistral-24B, Qwen3-32B, Llama3.3-70B under benchmarks including MMLU, WMDP and ARC.

Critical strengths
  • The problem of study in this paper is critical and relevant. As AI agents' capability continues to grow, the risk of sandbagging is real, and research on capability elicitation techniques is timely and relevant.
Critical weaknesses
  • (Major) The methodology contains a number of design choices, some of which are not justified well or even carefully stated. To name a few:
    • "Communities" need to be formed using graph clustering, but although "functional networks" are already are graphs, the authors used a separately constructed graph, since "thresholding leaves some heads with no edges, and such heads form singleton communities that give no information." I find such justification not convincing, and the construction of the separate graph is not even stated well.
    • Some magic numbers appear in the methodology. For example, top 25% similarity edges are included. This feels arbitrary and dataset-dependent, not universal.
  • (Major) The evaluation is questionable.
    • The baseline selection seems weak. The inference-level baselines or weight-level baselines considered are standard techniques that are not necessarily designed for interventions about LLM capabilities. Meanwhile, there does exist a broad literature on head-level interventions about LLM capabilities that the authors should have compared with, or at least discussed. To name a few:
      • Kayo Yin, Jacob Steinhardt. Which attention heads matter for in-context learning? arXiv:2502.14010
      • Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, Kun Wang, Yang Liu, Junfeng Fang, Yongbin Li. On the role of attention heads in large language model safety. ICLR 2025
      • Anna Bair, Yixuan Even Xu, Mingjie Sun, and J. Zico Kolter. Compressed Sensing for Capability Localization in Large Language Models. ICML 2026
    • (Minor) It is unclear to me, why GPT-OSS is in Table 1, but not in Table 2.
    • (Minor) It is also unclear to me what the error bars mean in the tables.
Questions for authors

Feel free to respond to the weakness raised.

6 comments
AuthorsOpenReview ↗

Official Comment by Authors

We appreciate the reviewer's feedback and their recognition of the importance and relevance of studying sandbagging as AI model capabilities continue to advance. We understand that the reviewer's main concerns relate to the justification of our methodological choices and the selection of evaluation baselines. Below, we address these concerns through additional experiments and clarifications, and hope the reviewer will consider updating their score.

Why Two Graphs?

We appreciate the reviewer's concern regarding the use of two graphs and agree that this methodological choice deserves a clear justification. Global thresholding at the top 25% most similar edges leaves 9.4%, 3.1%, 2.8%, and 9.2% of heads without any connections in the honest Mistral, Qwen, Llama, and GPT-OSS graphs, respectively. These isolated heads form singleton communities. Therefore, "within-community" connectivity has no intuitive meaning, and we cannot measure meaningful changes for these heads.

To preserve and standardize this notion of connectivity, we detect communities using a symmetric k-nearest-neighbor (kNN) graph constructed from the same honest-condition similarity matrix. Two heads are connected if either ranks the other among its kk nearest neighbors, a common practical approach also recommended by von Luxburg [1]. Communities are detected once and held fixed across conditions, ensuring that we are not selecting communities convenient for a particular setting or measurement. Importantly, all connectivity measurements and head-selection scores are computed on the original 25%-density graphs; the kNN graph supplies only community labels.

We further test sensitivity to the kNN neighborhood size, varying kk from 0.3k00.3k_0 to 4k04k_0, where k0k_0 is the default neighborhood size. As shown in the table below, head-score rankings remain strongly correlated with those at k0k_0 (Spearman ρ=0.842\rho = 0.842–0.9720.972), supporting robustness to neighborhood size. Full details are provided in Appendix D.

Table: Spearman correlation between head scores at each kNN neighborhood size and the default k0k_0. Rankings remain strongly correlated across all tested neighborhood sizes and models.

Model (k0k_0)0.3k00.3k_00.5k00.5k_02k02k_04k04k_0
Mistral-24B (20)0.8930.9180.9300.921
Qwen3-32B (50)0.9550.9720.9610.933
Llama-3.3-70B (10)0.8620.9010.9670.944
GPT-OSS-20B (50)0.8420.8530.8530.877

k% Thresholding

In functional networks, thresholding retains the strongest statistical dependencies, allowing network structure to be studied without retaining numerous weak connections. Most prior functional brain-network studies use densities within 0.15–0.35; we use the middle of this range, 0.25, as the density. We performed sensitivity tests to demonstrate robustness regardless of the chosen density, which can be found in Appendix C. We report that varying the density yields head rankings that remain highly correlated with those from a density of 0.25 (Spearman = 0.828–0.960).

Here, we provide additional empirical evidence of robustness by evaluating GraphLearned across densities of 0.20–0.35. Performance remains comparable across both models and all three benchmarks, with mean accuracy differing by at most 1.90 percentage points from the default density of 0.25. This demonstrates that performance is a result of the GraphLearned algorithm, rather than an artifact of density selection.

Table: GraphLearned accuracy (%) across graph densities on MMLU, WMDP, and ARC. Performance remains very close to the default density of 0.25, demonstrating robustness to the choice of graph density.

ModelDensityMMLUWMDPARC
Mistral-24B0.2070.6060.6088.05
0.2571.3560.7089.85
0.3069.8061.0587.95
0.3571.1060.9089.15
Qwen3-32B0.2069.2562.0890.67
0.2570.2563.6089.10
0.3069.7564.4290.33
0.3569.4263.8389.67

Reference

[1] Ulrike von Luxburg. A Tutorial on Spectral Clustering. Statistics and Computing, 17(4):395–416, 2007.

AuthorsOpenReview ↗

Official Comment by Authors

Missing Baselines

We compare our method with the requested baselines, namely SAHARA (Zhou et al.), compressed sensing (Bair et al.), and FV (Yin & Steinhardt) on Mistral-24B and Qwen3-32B. We find that our method outperforms all three baselines across both models and all benchmarks.

We would like to note that, in their original formulations, these requested baselines were not specifically designed to elicit suppressed capabilities from sandbagging models, which our work studies. This may help explain the poor performance of some baselines, such as SAHARA, on this task. Furthermore, we find that some of these interventions degrade honest performance. For example, FV reduces Mistral-24B's honest accuracy on WMDP by 7.5 percentage points. We have added a discussion of these methods to the Related Work section of our updated manuscript.

Table: Accuracy (%) on MMLU, WMDP, and ARC. Our intervention achieves higher capability elicitation than the benchmarked baselines across both models.

ModelMethodMMLUWMDPARC
Mistral-24BSAHARA25.0024.2517.00
Compressed sensing66.7557.5085.25
Function vectors (FV)69.7556.2587.50
Ours (GraphLearned)71.3560.7089.85
Qwen3-32BSAHARA28.2528.2517.50
Compressed sensing61.5059.2573.00
Function vectors (FV)51.0050.0063.25
Ours (GraphLearned)70.2563.6089.10

Most importantly, we would also like to highlight that the baselines considered in our manuscript, namely response prefilling, few-shot prompting, noise injection, and SFT with few demonstrations, are well-established capability elicitation approaches, as benchmarked in prior work [1–3].

GPT-OSS-20B Results

GPT-OSS-20B follows the same trend as the other models. Graph-informed methods improve capability elicitation over their corresponding baselines. We have added these results to Table 2 of our updated manuscript.

ModelMethodMMLUWMDPARC
GPT-OSS-20BPrefill42.2533.2552.25
Ours + Prefill59.2550.0075.00
Few-shot51.3348.3361.08
Ours + Few-shot56.0850.5067.75
Noise Injection45.90 ± 8.1942.35 ± 6.9153.25 ± 14.87
SFT (LoRA)60.10 ± 6.4054.10 ± 3.1278.90 ± 8.34
SFT (Full)62.40 ± 7.3557.30 ± 2.4475.65 ± 11.28
Ours (GraphLearned)63.70 ± 4.2157.45 ± 3.1681.05 ± 5.08

Meaning of Error Bars

In Tables 2 and 3, ± denotes the sample standard deviation of accuracy across five independent trials, each corresponding to a separately seeded run of the weight-level method under the same evaluation setup. Inference-level methods use fixed prompts and fixed few-shot demonstrations, without independent training trials. Therefore, the same trial-level standard deviation is not applicable.

We once again thank the reviewer for their engagement with our work. We believe that our work contributes an original approach to studying sandbagging through attention-head functional networks, yielding meaningful empirical gains over existing capability elicitation baselines, with potential applications to other AI safety problems. We hope our additional experiments and clarifications have addressed the concerns raised and sincerely hope the reviewer will consider raising their score.

Reference

[1] Felix Hofstätter, Teun van der Weij, Jayden Teoh, Rada Djoneva, Henning Bartsch, Francis Rhys Ward. The Elicitation Game: Evaluating Capability Elicitation Techniques. ICML 2025.

[2] Cameron Tice, Philipp Alexander Kreer, Nathan Helm-Burger, Prithviraj Singh Shahani, Fedor Ryzhenkov, Fabien Roger, Clement Neo, Jacob Haimes, Felix Hofstätter, Teun van der Weij. Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models. NeurIPS 2025.

[3] Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov, David Krueger. Stress-Testing Capability Elicitation With Password-Locked Models. NeurIPS 2024.

I thank the authors for the detailed response. I have gone through it. I believe it answers some questions of mine about error bars and GPT-oss results. The authors also have run the attention-head intervention baselines I mentioned, which is definitely a good step. The response has indeed improved my view of this manuscript.

However, given the paper in its current form, I would still not recommend acceptance for it. Even after reading the response, I do not find the methodology more convincingly justified, and some of my concerns about the evaluation remain. I therefore would continue to recommend weak rejection.

Reviewer_fzmTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how sandbagging affects the internal structure of LLMs by constructing functional graphs based on attention-head activations. The authors find that within-community connectivity decreases during sandbagging and use these changes to identify attention heads for targeted interventions. They show that zero ablation of the selected heads can recover suppressed capabilities and propose GraphLearned, a lightweight trainable intervention that achieves promising results compared to existing capability elicitation methods. I find results intriguing and interesting, although have some concerns on how isolated they are.

Critical strengths
  • The paper introduces a graph-based approach to studying sandbagging by analyzing functional relationships between attention heads. This offers an interesting direction for understanding changes in the internal computations of LLMs.

  • The authors evaluate their approach across multiple model architectures, community detection algorithms, and benchmarks. The consistent reduction in within-community connectivity across these settings is an interesting empirical finding.

  • The proposed graph-based head selection consistently outperforms random-head ablation baselines, while largely preserving honest performance. Furthermore, GraphLearned achieves competitive or superior results compared with several existing elicitation methods, suggesting practical potential for the approach.

Critical weaknesses
  1. The paper compares graph-selected heads against randomly selected heads under zero ablation, and GraphLearned against existing fine-tuning methods. However, it does not directly compare GraphLearned against zero ablation on the same identified heads under matched experimental conditions. It also does not evaluate GraphLearned on randomly selected heads. I believe a systematic comparison of all combinations of head selection (graph-selected versus random) and intervention (zero ablation versus GraphLearned) is necessary to establish the individual contributions of the graph-based selection and the learned transformation.

  2. The paper shows that setting the selected attention-head outputs to zero can substantially recover performance, while barely affecting honest accuracy. However, I would like to understand why this works. Are these heads particularly important for enforcing sandbagging, or do they simply contribute less to the model's normal computations? For example, do the selected heads have smaller activation magnitudes or smaller contributions after the output projection than other heads? Comparing these quantities with those of randomly selected heads would help explain why zeroing them improves performance without significantly affecting honest behavior.

  3. The paper shows that connectivity decreases when a model is instructed to sandbag. However, this decrease might simply reflect the model operating under a different behavioral objective rather than a mechanism specific to capability suppression. For example, would a similar decrease occur if the model were instructed to refuse to answer questions, produce random answers, or otherwise deviate from its usual question-answering behavior? Such control experiments would help determine whether the observed structural changes are genuinely associated with sandbagging or represent a more general response to changes in model behavior.

  4. The authors state that hyperparameters are tuned on 400 held-out MMLU questions, but it is unclear how the graph-construction and head-selection hyperparameters are chosen. In particular, are these parameters tuned to maximize capability recovery after zero ablation, or are they selected independently of downstream performance? If they are optimized for ablation performance, were the random-ablation baselines given comparable tuning opportunities? Maybe I don't understand well, but this is a major concern to me. Is it possible the improve in performance is due to the the tuning compensating for the ablation and the sandbagging, and not due the the ablation (or graphlearn) itself?

  5. Just a minor concern. This method would be very hard to implement in proprietary models where weights are unknown.

Questions for authors

I have many questions in the weaknesses section.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for their thoughtful feedback and for recognizing the novelty of our attention-head functional network approach to studying sandbagging, its consistent empirical findings across models and benchmarks, and its effectiveness in recovering suppressed capabilities. Below, we provide additional experiments and clarifications to address the questions raised.

Zero Ablation vs. GraphLearned: Random and Graph-Selected Heads

We compare zero ablation and GraphLearned using random and graph-selected heads under prompted sandbagging on Mistral-24B and Qwen3-32B. We find that graph-selected head ablation consistently outperforms random ablation across all datasets for both zero ablation and GraphLearned experiments. Adding learnable parameters further improves performance. These results further support the benefits of graph-based head selection.

Table: Zero ablation versus GraphLearned under random and graph-based head selection. Values are final-evaluation accuracy (%). Error terms denote sample SD across five head sets for random ablation, five head-set means for random GraphLearned, and five training trials for graph-selected GraphLearned. Graph-selected ablation uses one fixed head set.

ModelMethodMMLUWMDPARC
Mistral-24BZero ablation (random)22.40 ± 2.4622.15 ± 2.3914.05 ± 3.26
Zero ablation (graph)53.0045.5060.50
GraphLearned (random heads)55.11 ± 5.7649.97 ± 2.6064.13 ± 8.73
GraphLearned (graph-selected heads)71.35 ± 2.8060.70 ± 0.8289.85 ± 3.21
Qwen3-32BZero ablation (random)27.05 ± 4.1928.10 ± 6.2916.90 ± 6.37
Zero ablation (graph)64.7561.2581.50
GraphLearned (random heads)49.65 ± 4.3347.62 ± 3.8952.05 ± 9.98
GraphLearned (graph-selected heads)70.25 ± 1.3363.60 ± 1.1089.10 ± 3.18

Why Does Graph-Localized Ablation Work?

We appreciate the reviewer's insightful question about why graph-localized ablation recovers suppressed capabilities while largely preserving honest accuracy. Our additional experiments show that graph-selected heads are not simply low-magnitude components. We find that their activation magnitudes and projected-output contributions are comparable to those of random heads from the same layers, and sometimes larger. Furthermore, random heads matched in layer and magnitude fail to achieve comparable capability recovery.

To better understand the intervention's effects, we examine changes in the model's internal representations. We find that progressively ablating graph-selected heads shifts sandbagging representations toward the honest condition along the honest–sandbagging contrast direction, accompanied by greater recovery in sandbagging accuracy. These effects are largely absent under matched random ablation. Full experimental details are provided in Appendix H.

AuthorsOpenReview ↗

Official Comment by Authors

Non-Sandbagging Prompt Changes

To study whether the observed reduction in within-community connectivity is specific to sandbagging or simply reflects a change in prompting, we repeat our analysis under six alternative non-sandbagging prompts: an honest paraphrase (Appendix B), a teacher persona, an educational context, a reasoning instruction, random answering, and refusal. Each is evaluated as a separate prompting condition, using the same community assignments identified under honest prompting.

We measure the mean fraction of each head's connections that remain within its community and compare it with the original honest condition. As shown in the table below, sandbagging reduces this fraction by 4.5–11.7 percentage points, whereas the alternative prompts produce much smaller changes, ranging from −1.9 to +0.4 percentage points. These results show that the reduction in within-community connectivity under sandbagging is substantially larger than that induced by other changes in prompting.

Table: Mean fraction of each head's connections within its community across prompting conditions. Parentheses indicate changes relative to honest prompting (percentage points). Sandbagging produces substantially larger reductions than the alternative prompts.

ModelConditionLouvainLeidenInfomapSpectral
Mistral-24BHonest (H).337.333.462.388
Sandbagging (S).254 (−8.3).253 (−8.0).345 (−11.7).312 (−7.6)
Honest paraphrase (H₂).336 (−0.1).330 (−0.3).456 (−0.6).387 (−0.2)
Persona.335 (−0.1).331 (−0.2).461 (−0.1).389 (+0.1)
Context.338 (+0.1).334 (+0.2).465 (+0.3).392 (+0.4)
Reasoning.335 (−0.2).329 (−0.4).457 (−0.4).388 (−0.1)
Random answer.325 (−1.2).321 (−1.1).447 (−1.5).379 (−0.9)
Refusal.328 (−0.8).325 (−0.8).452 (−1.0).381 (−0.8)
Qwen3-32BHonest (H).357.355.634.512
Sandbagging (S).309 (−4.7).304 (−5.2).589 (−4.5).465 (−4.6)
Honest paraphrase (H₂).340 (−1.6).339 (−1.7).615 (−1.9).493 (−1.9)
Persona.354 (−0.3).352 (−0.3).637 (+0.3).511 (−0.1)
Context.354 (−0.3).354 (−0.2).633 (−0.1).511 (−0.1)
Reasoning.349 (−0.7).349 (−0.6).632 (−0.2).507 (−0.5)
Random answer.352 (−0.4).350 (−0.5).635 (+0.1).510 (−0.1)
Refusal.344 (−1.3).341 (−1.5).624 (−1.0).503 (−0.9)

Hyperparameter Tuning

We clarify that our graph-construction, clustering, and thresholding settings are chosen independently of downstream task performance. Only the percentage of selected heads is tuned on held-out MMLU validation questions. Our original random controls match this percentage to isolate the effect of head selection at a fixed ablation budget. However, we agree that independently optimizing the random-ablation budget provides a useful additional comparison.

We therefore also optimize the random-ablation budget using the same validation procedure. As shown in the table below, random ablation selects 15% of heads for both models, compared with 2% and 3% for graph selection on Mistral and Qwen, respectively. Despite using substantially fewer heads, graph selection outperforms the optimized random baseline by 10.35–29.60 percentage points across all six comparisons.

Table: Accuracy (%) under random and graph-based head selection with independently optimized ablation budgets. Random results report mean ± standard deviation across five head-selection seeds.

ModelSelectionHeadsMMLUWMDPARC
Mistral-24BRandom15%36.45 ± 12.0935.15 ± 9.1533.15 ± 18.32
Graph2%53.0045.5060.50
Qwen3-32BRandom15%44.60 ± 13.8243.15 ± 14.1051.90 ± 23.62
Graph3%64.7561.2581.50

Furthermore, GraphLearned on random heads receives independent hyperparameter tuning but remains outperformed by GraphLearned on graph-selected heads, supporting the contribution of graph-based localization beyond training alone. We have clarified the hyperparameter selection procedure and added this comparison to the revised manuscript.

We once again thank the reviewer for their thorough review. We hope that our additional experiments and clarifications have addressed the questions raised. We would appreciate your consideration of an updated score.

Thank you for the detailed rebuttal and additional experiments. I appreciate that the authors have addressed most of my concerns, particularly through the factorial comparison, non-sandbagging prompt controls, and improved random-ablation baselines. While the underlying mechanism of capability recovery remains only partially explained, the additional evidence substantially strengthens the paper. I would encourage the authors to incorporate the most important findings, particularly the factorial comparison, into the main text, while keeping the detailed analyses in the appendix. Overall, the rebuttal has improved my assessment of the paper. I lean towards "weak accept" since it is hard to assess the new state of the manuscript.

AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for their positive assessment and for acknowledging that our rebuttal has addressed most of their concerns and that the additional evidence has substantially strengthened the paper. We will incorporate the reviewer's suggestions into our manuscript.

Best,

Authors

Reviewer_aVCCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies what changes inside an LLM when it deliberately underperforms. The authors build a functional graph of attention heads, where each head is a node, and two heads are connected if their activations vary together across 400 MMLU questions. Comparing honest and sandbagging prompts across four models and four community detection methods, they find that heads keep fewer connections within their own group when the model sandbags.

They then use this to recover the hidden ability in two ways. First, switching off the heads that lose the most connections improves accuracy under sandbagging more than switching off random heads. Combining this with prefill or few-shot prompting improves those methods further. Second, they propose GraphLearned, which trains a small adjustment on only the selected heads from a few correct examples while the rest of the model stays frozen. GraphLearned outperforms noise injection, LoRA, and full fine-tuning with far fewer trained parameters and less variance.

Critical strengths
  1. Using network neuroscience tools to study sandbagging is original. The connectivity drop appears in all four models and with all four community detection methods. The head ablation result beats three sensible random controls in every case (global, layer-matched, and degree-matched).
  2. GraphLearned trains very few parameters, matches or beats full SFT, and has much lower variance.
Critical weaknesses
  1. The weight level gains may come from tuning any small set of heads, not from using the graph to choose which ones. GraphLearned should be compared against the same method trained on randomly chosen heads.
  2. There is no comparison against picking heads by a simpler score, such as how much each head’s activation changes between honest and sandbagging runs.
  3. Gains are often negligible and come from single runs. Qwen prefill goes from 67.75 to 68.00 (Table 2), and most Table 3 inference cells change by 0–0.5pp.
  4. On the Mistral base model, prefill alone (57.0) beats GraphLearned (52.3).
  5. Llama 70B has no full SFT result in Table 2
Questions for authors
  1. What happens when GraphLearned is trained on random heads?
  2. How does graph based selection compare with selecting heads by their direct activation change between conditions?
  3. Does a non sandbagging prompt change also reduce within community connectivity?
  4. Can you add accuracy errors to Table 2 and Table 1 and the inference level results in Table 3?
3 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for their thoughtful engagement with our work and for recognizing the originality of applying tools from network neuroscience to study sandbagging, as well as the breadth of our evaluation across models and datasets. Below, we address each of the reviewer's questions and present additional experimental evidence.

GraphLearned on Random Heads

The reviewer's concern is valid as the gains from GraphLearned may arise from the learned intervention rather than graph-based head selection. To examine this, we additionally train GraphLearned on five independently sampled random head sets, each containing the same number of heads as our graph-selected set. We conduct this comparison under prompted sandbagging on Mistral-24B and Qwen3-32B, evaluating on MMLU, WMDP, and ARC.

As shown in the table below, GraphLearned on graph-selected heads consistently outperforms GraphLearned on random heads across both models and all three benchmarks, with improvements of 10.73–37.05 percentage points. These results demonstrate that graph-based head selection contributes substantially to capability recovery beyond the learned intervention itself.

Table: GraphLearned on random versus graph-selected heads. Accuracy (%) under prompted sandbagging. Graph-selected heads consistently outperform random heads.

ModelMethodMMLUWMDPARC
Mistral-24BGraphLearned (random heads)55.11 ± 5.7649.97 ± 2.6064.13 ± 8.73
GraphLearned (graph-selected heads)71.35 ± 2.8060.70 ± 0.8289.85 ± 3.21
Qwen3-32BGraphLearned (random heads)49.65 ± 4.3347.62 ± 3.8952.05 ± 9.98
GraphLearned (graph-selected heads)70.25 ± 1.3363.60 ± 1.1089.10 ± 3.18

Comparison with Activation-Based Head Selection

We compare our graph-based head selection with an alternative approach that selects heads based on their normalized activation changes between honest and sandbagging conditions. We use equal head budgets and apply identical zero-ablation interventions. We evaluate both approaches across nine sandbagging model settings, including five additional sandbagging-trained models from Greenblatt et al. [1] and Taylor et al. [2].

As shown in the table below, graph-based selection achieves higher capability elicitation accuracy across all nine settings, with improvements of up to 50 percentage points on Willow and 32.43 percentage points on circuit-broken Mistral Instruct. These results demonstrate that graph-based selection identifies more effective heads for capability elicitation than selection based solely on activation changes.

Table: Capability elicitation accuracy (%) using activation-based versus graph-based head selection under identical head-ablation budgets.

Sandbagging ModelActivation (%)Graph (%)Graph − Activation (pp)
Redwood DeepSeek-Math-7B/Pythia-1B7.2910.94+3.65
Gemma-2-9B Code25.0027.34+2.34
Willow36.7286.72+50.00
Tarun57.3059.12+1.82
Cedar73.4780.61+7.14
Password-locked Mistral Base56.4957.57+1.08
Password-locked Mistral Instruct50.0051.35+1.35
Circuit-broken Mistral Base59.4661.08+1.62
Circuit-broken Mistral Instruct4.3236.76+32.43

References

[1] Ryan Greenblatt, Fabien Roger, Dmitrii Krasheninnikov, David Krueger. Stress-Testing Capability Elicitation with Password-Locked Models. NeurIPS 2024.

[2] Jordan Taylor, Sid Black, Dillon Bowen, Thomas Read, Satvik Golechha, Alex Zelenka-Martin, Oliver Makins, Connor Kissane, Kola Ayonrinde, Jacob Merizian, Samuel Marks, Chris Cundy, Joseph Bloom. Auditing Games for Sandbagging. arXiv:2512.07810, 2025.

AuthorsOpenReview ↗

Official Comment by Authors

Non-Sandbagging Prompt Changes

To study whether the observed reduction in within-community connectivity is specific to sandbagging or simply reflects a change in prompting, we repeat our analysis under six alternative non-sandbagging prompts: an honest paraphrase (Appendix B), a teacher persona, an educational context, a reasoning instruction, random answering, and refusal. Each is evaluated as a separate prompting condition, using the same community assignments identified under honest prompting.

We measure the mean fraction of each head's connections that remain within its community and compare it with the original honest condition. As shown in the table below, sandbagging reduces this fraction by 4.5–11.7 percentage points, whereas the alternative prompts produce much smaller changes, ranging from −1.9 to +0.4 percentage points. These results show that the reduction in within-community connectivity under sandbagging is substantially larger than that induced by other changes in prompting.

Table: Mean fraction of each head's connections within its community across prompting conditions. Parentheses indicate changes relative to honest prompting (percentage points). Sandbagging produces substantially larger reductions than the alternative prompts.

ModelConditionLouvainLeidenInfomapSpectral
Mistral-24BHonest (H).337.333.462.388
Sandbagging (S).254 (−8.3).253 (−8.0).345 (−11.7).312 (−7.6)
Honest paraphrase (H₂).336 (−0.1).330 (−0.3).456 (−0.6).387 (−0.2)
Persona.335 (−0.1).331 (−0.2).461 (−0.1).389 (+0.1)
Context.338 (+0.1).334 (+0.2).465 (+0.3).392 (+0.4)
Reasoning.335 (−0.2).329 (−0.4).457 (−0.4).388 (−0.1)
Random answer.325 (−1.2).321 (−1.1).447 (−1.5).379 (−0.9)
Refusal.328 (−0.8).325 (−0.8).452 (−1.0).381 (−0.8)
Qwen3-32BHonest (H).357.355.634.512
Sandbagging (S).309 (−4.7).304 (−5.2).589 (−4.5).465 (−4.6)
Honest paraphrase (H₂).340 (−1.6).339 (−1.7).615 (−1.9).493 (−1.9)
Persona.354 (−0.3).352 (−0.3).637 (+0.3).511 (−0.1)
Context.354 (−0.3).354 (−0.2).633 (−0.1).511 (−0.1)
Reasoning.349 (−0.7).349 (−0.6).632 (−0.2).507 (−0.5)
Random answer.352 (−0.4).350 (−0.5).635 (+0.1).510 (−0.1)
Refusal.344 (−1.3).341 (−1.5).624 (−1.0).503 (−0.9)

Error Bars

For weight-level methods in Tables 2 and 3, we report mean accuracy and sample standard deviation across five independent training trials. For inference-level methods, we use fixed prompts and few-shot demonstrations, without repeated training trials. Therefore, the same trial-level standard deviation is not applicable. Similarly, the within-community connectivity values in Table 1 are computed from fixed functional graphs rather than independent training trials.

Missing Llama-70B Full SFT

Due to the academic compute resources available to us, we could not run full SFT on Llama-70B. Our GPUs were able to accommodate the smaller Mistral-24B and Qwen3-32B, for which we reported the full-SFT comparisons. We have also included full-SFT results for GPT-OSS-20B in our updated manuscript.

We once again thank the reviewer for their feedback on our work. We hope that our additional experimental results and clarifications have addressed the concerns raised, and we would be grateful if the reviewer would consider updating their score accordingly.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer aVCC,

Thank you once again for your feedback on our work. In our rebuttal, we’ve responded to the points you raised. As the author–reviewer discussion period closes very soon, we’d greatly appreciate your thoughts on whether our responses address your concerns. Please let us know if there’s anything else we can clarify.

Thanks,

The authors

Comments on the paper

1 comment
Area_Chair_odgKOpenReview ↗

Reminder to Reviewers: Please Engage in the Discussion

Dear Reviewers,

As the authors have now submitted their rebuttal, I would kindly ask you to read their responses and engage in the discussion.

Thank you!

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.