acceptodds
Under review as a conference paper at ICLR 2027

Benchmarking Interpretability Localizers For Adaptation Placement

Abstract

Parameter-efficient fine-tuning makes it cheap to adapt a language model to a shifted distribution, but not to decide which layers should receive the update. Mechanistic-interpretability localizers—sparse autoencoders, attribution patching, gradient and Fisher saliency, drift-native gradient directions, linear probes—promise "where," and that promise is a falsifiable prediction: if a localizer identifies where the shifted computation lives, adapting those layers should beat adapting anywhere else at the same budget. We build the ground truth to test it: dense grids of matched-budget LoRA adapters at every candidate layer window, across four domains and three model families (Gemma-2 9B; Qwen-2.5 7B and Llama-3.1 8B), each grid measuring which placements actually recover the task. Scored against the grids under a frozen protocol, seven localizers—including AdaLoRA, a method built to allocate adaptation budget—yield one verdict: no localizer reliably predicts where to adapt. Where placement is certifiably flat, every localizer ties a random matched-budget window; where placement demonstrably matters, even the localizers that name the peak confer no certified advantage over a random window; a depth band frozen on two model families breaks on the third; and no windowed placement, localizer-chosen or optimal, recovers whole-network performance. The negative is bounded, not merely unresolved: no tie conceals a localizer advantage above +5.7pp (95% confidence). We release PlacementBench—the grids, the frozen scoring-and-admissibility protocol, and effect bounds on every tie—as an instrument rather than an indictment: a future localizer overturns this result by clearing its two bars, identification and certified advantage. A first use: a cross-cell readout calibrated on the graded grids identifies a top-2 window in 8/9 cells and clears the certification bar on one, where every per-cell rule fails.

Primary area:
datasets and benchmarks
Keywords:
parameter-efficient fine-tuning, LoRA, adaptation placement, mechanistic interpretability, sparse autoencoders, attribution patching, Fisher information, linear probes, AdaLoRA

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JdeaOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors study to what extent localizers, methods that identify where, e.g. in which layers, in a model a desired task performance is located. This is motivated by the need to decide where parameter-efficient fine-tuning should be placed for optimization. The authors propose a benchmark that tests to what extent localizers are able to uncover the best placement of a parameter update. This benchmark is then used for a study across seven localizers, three model families and four domains (code, Json, Q&A, regex).

Critical strengths
  • The reframing of capability-drift adaptation as a benchmark question is relevant and interesting.
  • The definition of appropriate baselines (random, uniform) is appreciated and grounds the overall analysis.
  • The empirical study covers a reasonable range of models, tasks and localizers.
Critical weaknesses
  • The study assumes that localization under the investigated tasks is linked to an identifiable location that is optimal for parameter-efficient finetuning, but localization as identified by localization methods do not necessarily correspond to where the parameter updates should appear. Localizers may instead correspond to layers where relevant computations happen, but not where they originate.

  • The paper’s results and methods are at times difficult to parse. The authors use condensed language and jargon to describe simple results, e.g. “Headroom is ample (H = 33.3pp, z= 8.8, R= 11.7: whole-network reaches 0.580 over the 0.247 no-T2 floor, n= 300), so the flatness is informative.“ (l.246 - 247). Some language like “flat grid” vs “peaked grid”, (e.g “The code grid is flat” (l. 244)) are not sufficiently well introduced, making it difficult to understand what precisely is meant. See also questions below.

  • Presentation of results overall is not clear, e.g. Figure 1, rightmost panel: What does the hatching pattern on the rightmost panel indicate? It may refer to “QA panel grayed, ungraded” in the caption, but it is easily confused with “Gray shading = hook-reachable zone”.

  • Gradient localization (AtP) has been shown to be a noisy approximation [Jaf25], which can be robustified through application of selected propagation rules (see [Jaf25]). These have been shown to support downstream applications like circuit localization [Aro26a] and automated circuit description [Aro26b], and thus would be a relevant data point to stress-test the benchmark on a state-of-the-art localization method.

References - [Jaf25] FR Jafari et al. RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching, Mechanistic Interpretability Workshop, NeurIPS 2025 - [Aro26b] Arora, A., et al. Language model circuits are sparse in the neuron basis. arXiv preprint arXiv:2601.22594 (2026). - [Aro26b] Arora, A., et al. . ADAG: Automatically describing attribution graphs. COLM 2026

Questions for authors
  • What is “an offset ladder across depth”? (l. 154)
  • What does “hook-reachable zone” mean, why is the hook not always “reachable”? (see caption Fig. 1)
4 comments
AuthorsOpenReview ↗

Response to Reviewer Jdea (1/2)

Thank you for the concrete suggestions, including the RelP stress test. We have made the computation-versus-update distinction the organizing question and added the requested comparison. The common response summarizes the two new studies.

W1. Computation location need not identify update location. We agree; the revision tests this link rather than assuming it. The new adaptation-oriented selection experiment provides a positive case: target-calibrated selection improves PIQA over the random mean, and target-domain calibration beats cross-domain transfer on both PIQA and ARC-Challenge. The latter need not imply a random-baseline advantage, as ARC-Challenge illustrates.

The complete-window study separately tests attribution agreement and update performance. Exact activation patching selects the same window as AtP (layers 20-28, 22/40), showing that this choice does not depend on the gradient approximation. But changing the measured activations from attention outputs to MLP outputs or residual inputs moves all three methods to layers 33-41 (16/40). The choice of what to measure is therefore part of the placement recommendation. Three no-signal controls at the same parameter count score 17-18/40 and are not statistically separated from either attribution choice. Together, these results support evaluating the signal, its layer-selection rule and the task, rather than assuming computation location determines useful update location.

W2. Condensed language: H, z, R and "flat" / "peaked". We rewrote the sentence you quoted in terms of the observed accuracy range, all-layer reference, accuracy before adaptation to the shifted task and sample size. Section 3.4 and the glossary define H, z and R as summaries of adaptation gain and uncertainty; neither ratio is presented as a paired test or a bound on other placements. Main result paragraphs give numeric ranges instead of relying on "flat" or "peaked". Where these short labels remain in the landscape-summary table (Table 3) and appendix matrix (Table 5), the main caption explains that they describe the tested curves, not statistical equivalence or a global optimum.

W3. Figure 1 hatching and shading. The rewritten caption distinguishes them. Full-panel hatching means QA is not graded. The two grey bands show which windows the layer-20 measurements can recommend: the darker band ends at center 18 for the SAE top-five readout; the lighter band ends at center 22 for the AtP+GradDrop rule based on the end of a high-score band. They restrict what the measurement can recommend, not which layers can be trained. Error bars are marginal binomial standard errors, not uncertainty in paired differences, and do not account for repeated records or training/evaluation overlap.

AuthorsOpenReview ↗

Response to Reviewer Jdea (2/2)

Response to Reviewer Jdea: Part 2 of 2

W4. RelP as a stronger localization method. We added RelP (Rezaei Jafari et al., 2025) to the complete 34-window study in Section 4.8, Figure 2 and Appendix B. AtP, RelP and exact activation patching use the same ten development pairs, continuation target and attention-output locations across all 42 layers. We adapted the authors' TransformerLens fork for the merged bf16 Gemma-2 9B, using the LN, Identity and Half propagation rules without AH. This is an adapted implementation, not an unmodified reproduction of their pipeline; Appendix B defines these choices and distinguishes attribution from adaptation.

RelP selects layers 33-41 (16/40), whereas AtP and exact patching select 20-28 (22/40). RelP minus AtP is -15.0 percentage points, with task-level 95% interval [-25.0, -5.0] and exact McNemar p = 0.0703. Accounting for the eight construction groups gives interval [-35.0, +2.5] and sign-flip p = 0.375. The direction favors AtP, but the exact test does not establish superiority. The secondary reference negative log-likelihood, which measures how much probability the model assigns to the reference answer, favors RelP instead: difference -0.00556 (lower is better), task interval [-0.00877, -0.00229], construction interval [-0.02191, 0.01042], sign-flip p = 0.539. Thus the behavioral and likelihood endpoints need not agree.

We also report how closely the methods match exact patching. On the ten development pairs, mean per-pair Pearson correlations of attention-output layer profiles are 0.838 for AtP and 0.651 for RelP's forward-exact variant. Here exact patching inserts clean activations into the counterfactual run. The variant preserves the ordinary activation's bf16 forward value while retaining the Identity rule's backward coefficient; Appendix B defines it. These are attribution-agreement measures, not correlations with adaptation accuracy.

Leaving out individual development pairs can move AtP to RelP's window and the primary RelP selector to layers 23-31; the forward-exact variant retains 33-41 in these checks. We report these sensitivities rather than claiming RelP is generally worse. The study directly tests whether its circuit-discovery improvements translate to selecting useful update locations, without assuming that they must.

Q1. "Offset ladder across depth". We removed the term. Section 3.1 describes a separate five-layer window sweep with centers {14, 18, 20, 22, 26, 30, 34} and a smaller parameter budget, not extra points on the nine-layer curve.

Q2. "Hook-reachable zone". We replaced the phrase with an explanation of where activations are captured. A hook is that capture point. In the original SAE and AtP+GradDrop configurations it is at residual layer 20, so later layers cannot contribute to that captured activation. Averaging five distinct indices from layers 0-20 cannot produce a center above 18; the band-end rule cannot name a center above 22. These limits apply to the readout, not the trainable layers. Sections 3.2-3.3, Table 1 and Figure 1 now explain this, and the all-layer hook sweep tests capture points 0-41.

Reviewer_JdeaOpenReview ↗

Reply

I appreciate the provided rebuttal, additional evaluations on RelP and clarifications. As also noted by the other reviewers, the paper suffers from limited readability and lacks a sufficiently strong motivation for its main claim. Unfortunately, these issues cannot be adequately addressed within the scope of the rebuttal. I will maintain my score and continue to follow the discussion.

Thank you for considering the additional evaluations and clarifications. We would like to clarify that the revised manuscript itself, beyond the rebuttal comments, rewrites the motivation in the Introduction, replaces the quoted headroom passage with numeric accuracy comparisons, and presents the new evaluations in Sections 4.8–4.9. Could you clarify what remains insufficient in the revised motivation: the importance of testing whether localization guides adaptation, or the contribution established by our results? This would help us understand the remaining objection without repeating the technical rebuttal.

Reviewer_dehDOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper’s goal is to test how good interpretability techniques are at guiding where you should update models. They train LoRAs on a range of layers and then compare 7 different interpretability techniques (like SAEs, probes, Aatp*, etc.) to see whether they can find the best layers for training LoRAs. Their results on their benchmark PlacementBench show that localization according to these methods doesn’t necessarily indicate which layer is best for training LORAs.

Critical strengths
  • The idea of using interpretability techniques to choose which layers to train with LoRA could be interesting if it works.
Critical weaknesses
  • The paper is really hard to follow. It was difficult for me to understand all the details and results and to verify all the scientific claims and experimental setups.

  • I think the contribution of this paper is not significant enough. If the conclusion had been that interpretability methods offer a cheaper or faster way to identify the best layers for training LoRAs, that would have been interesting. However, the final conclusion is that those interpretability methods, which were not designed to solve such a task, are indeed unsuitable for it. In addition, some of the methods (for example, SAEs) can require time and compute, and it is not clear that there is a reason to use those techniques.

Questions for authors

I would be happy if you could explain more on each localization method at section 3.2, this section is critical and I think it could be better explained.

3 comments

Thank you. We took the readability concern seriously and revised the presentation to explain the question, each method and the practical comparisons before introducing technical details. The common response summarizes the new experiments.

W1 / Q1. Explain the localization methods. Section 3.2 now gives each of the seven methods its own numbered explanation: what it measures, how that signal is obtained and what it does not establish about useful update locations. For example, the SAE entries explain feature contributions, Fisher uses squared gradients, the gradient-direction method compares the directions of gradients between domains, and AdaLoRA uses the rank allocation learned during adaptation. The section also identifies where activations are captured and the model configurations used.

Section 3.3 separately explains how these measurements become a placement recommendation. Depending on the method, it averages the indices of the five layers with largest-magnitude scores, derives a center from a high-score band's endpoint or uses the first layer reaching 90% of the probe's maximum discrimination accuracy. It then explains mapping that recommendation to the nearest tested center, with a worked example and the limits imposed by capturing activations at layer 20. Section 4.8 explains AtP, RelP and exact patching using the same signal-to-selection structure.

W2. What is the practical contribution? The new experiments provide a positive example using a signal designed for adaptation. Target-calibrated selection improves PIQA by 2.38 percentage points over the mean of five random subsets (95% item interval [1.23, 3.50], Holm-adjusted p = 0.000122), with a positive difference in every seed average. The interval is conditional on the tested seeds and subsets. The three other benchmarks remain unresolved against random.

There is also a distinct benefit from calibrating on the target domain: it beats cross-domain transfer on PIQA and ARC-Challenge. This separates two practical questions: does the calibration domain matter, and does selection improve on random placement? ARC-Challenge answers the first positively without establishing the second. PIQA answers both positively. Its margin over the best seed-averaged random subset is 0.15 points descriptively, so we claim an advantage over the random mean, not established superiority over the best sampled subset.

Does this establish a cheaper or faster procedure? No. We now state this explicitly (Appendix B). The experiments compare accuracy at a fixed trainable-parameter budget; they do not measure a complete selection-plus-training time advantage. AtP and RelP were timed together with shared computation, so we cannot assign separate costs to them. The historical all-layer control (Section 5, Appendix A.12) outperforms the best window on both Llama code grids, but loses on Gemma-JSON and Qwen-regex. Because its numerical precision, adapter initialization, merging and per-layer rank also differ, those results do not isolate the benefit of distributing updates or establish an efficiency gain.

Why use expensive SAEs? Our findings do not establish that their cost is justified for placement selection in these settings. In the complete-window study, both SAE widths choose layers 14-22 (16/40), below the no-signal controls at 17-18/40 descriptively, not in a designated paired test. The original layer-20 measurement also restricts which windows their readouts can recommend. Section 4.3 shows that measuring later can let SAE-131k identify the JSON grid maximum, but the tested advantage over the best random draw remains unresolved after correction. This gives a concrete diagnosis of the configured method's limitation, rather than equating detection of a distribution shift with successful placement.

Statistical reporting. The original code/JSON comparisons retain paired exact tests with Bonferroni correction and the overlap limitations stated in the common response. The new studies report the specified comparisons, intervals and exact tests, with separate Holm corrections for the window/control tests and for the native random-baseline/domain-matching tests. The task-level intervals and the construction-level sensitivity answer different uncertainty questions; neither is used to override an inconclusive exact test. Full results are in Appendices B and C.

I appreciate the authors additional experiments and clarifications. They have addressed several of my requests, however, I believe the paper’s readability still requires substantial improvement beyond the scope of this rebuttal. I will therefore maintain my original score.

Thank you for revisiting the paper and acknowledging that several requests have been addressed. The revised manuscript itself includes numbered explanations of each localization method in Section 3.2 and separately explains how each signal becomes a placement recommendation in Section 3.3, with a worked example. These changes go beyond the explanations in our rebuttal comments. Could you identify one or two passages or sections in the revised manuscript that remain particularly difficult to follow? Concrete examples would help us understand and address the remaining readability concern.

Reviewer_faE1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PlacementBench for evaluating whether mechanistic interpretability methods can identify suitable layers for parameter-efficient adaptation of LLMs. The authors evaluate seven localizers using matched-budget LoRA placement experiments across multiple model architectures and distribution shifts. The research question is meaningful, and the extensive experiments and statistical analyses are valuable. This work provides a systematic negative conclusion as the contribution

Critical strengths

The paper addresses an important question concerning the relationship between interpretability and effective model adaptation, which is insufficiently investigated in the previous studies. The evaluations demonstrate a substantial experimental effort. The release of the placement grids and evaluation protocol could also provide a useful resource for future research.

Critical weaknesses

About Section 3.1. The primary evaluation relies on seven predefined window centers, but these candidate windows cover only a subset of possible layer placements. Whether the grid adequately characterizes the actual placement landscape. It is unclear whether the random placements are contiguous windows or independently sampled layer subsets.

Most evaluated localizers were not originally designed to predict optimal adaptation placement. Their failure to identify the best LoRA window does not necessarily indicate a failure of their underlying interpretability capabilities.

The main paper does not provide a sufficiently comprehensive and systematic presentation of the benchmarking results. Most experiments are provided in the appendix.

Questions for authors

It is suggested to clarify how the random placements are constructed, particularly whether they consist of contiguous layer windows or randomly selected layers. Given that the ground-truth optimum is identified from a limited set of candidate windows, is this search space sufficiently comprehensive to serve as a reliable ground truth for evaluating the localization accuracy of different methods?

Most evaluated localizers were not originally designed to predict optimal adaptation placement. How do the authors justify using LoRA placement performance as a measure of their localization capabilities? Could their failure to identify the best adaptation window reflect a mismatch between the evaluation objective and the original purposes of these methods?

3 comments

Thank you for recognizing the benchmark question and experimental effort. We have addressed the coverage concern with a complete-window study and made the distinction between interpretability and adaptation explicit. The common response summarizes the new results; below we address your specific points.

W1 / Q1. Window coverage and random placements. The original random placements are scattered subsets of nine distinct layers (seven on Qwen/Llama), not contiguous windows. Three draws are evaluated at the window budget. Section 3.1, Table 1 and the Figure 1 caption now state this directly.

We agree that seven centers cannot establish a global optimum. The paper describes the maximum as the largest value among the tested windows. The new study evaluates all 34 contiguous nine-layer windows in Gemma-2 9B under the same training and evaluation protocol (Section 4.8, Figure 2). Thus each selector is compared with the entire fixed-width window family, rather than a sampled subset. The best observed window is layers 24-32 (27/40), whose center 28 was absent from the original grid. This demonstrates why coverage matters; it does not revise the original maximum because the training horizon and evaluation differ. We label this maximum as selected after observing outcomes.

The equal-parameter controls also test non-contiguous placements. Separately, the native selection study compares against the mean of five random eight-layer subsets. Neither study claims to cover every possible subset of layers, and our method comparisons use specified pairs of alternatives rather than treating the best observed window as ground truth.

W2 / Q2. Is adaptation a fair test of interpretability methods? We agree that identifying influential layers need not identify where updates should go. The introduction now makes this the question being tested. We evaluate a signal together with its rule for choosing layers; a placement miss is not evidence that the underlying interpretability method fails at its original purpose.

The new results make this distinction concrete. First, exact activation patching and AtP select the same window, so that choice is not simply an artifact of AtP's gradient approximation. Appendix B also compares AtP and RelP with exact patching on the ten development pairs, keeping this attribution-agreement analysis separate from adaptation accuracy. Second, a signal designed for adaptation, Adaptive-LoRA's layer-intervention score, does improve PIQA over the random mean. Target-domain calibration also beats cross-domain transfer on PIQA and ARC-Challenge. The evidence therefore identifies circumstances in which selection helps, rather than supporting a blanket negative conclusion about localization.

W3. Results concentrated in the appendix. The main paper now presents both new studies: Section 4.8 / Figure 2 show all 34 windows, the attribution choices and equal-parameter controls; Section 4.9 / Table 2 report all four native primary outcomes with confidence intervals and p-values. The original Table 1, Figure 1, paired comparisons and robustness sections remain. Section 5 / Table 3 summarize the landscapes across three main model families and four domains. Appendices B and C provide the detailed window/control results, subset definitions, domain-matching tests and sensitivity analyses, including the separate single-seed comparison.

The expanded evaluation and revised presentation address several of my concerns. Some limitations remain regarding the scope and generalizability of the findings. After considering these revisions, I maintain my original assessment.

Thank you for revisiting the revised paper and acknowledging the improvements. To help us understand your remaining concerns, could you clarify which limitation is most decisive for your assessment: the breadth of the new evaluations, or the contribution supported by the results even within the tested settings? We recognize that the current evidence does not establish a general advantage for localization-guided adaptation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and new results

We thank the reviewers. The revision strengthens PlacementBench with a complete 34-window evaluation and evidence that task-calibrated layer selection can improve adaptation. Selection improves PIQA, one of four tested benchmarks, by 2.38 percentage points over the mean of five random layer subsets, and target-domain calibration beats cross-domain transfer on both PIQA and ARC-Challenge. These results help distinguish when localization guides useful updates from when it merely identifies influential computations.

The original contribution remains central: Table 1 and Figure 1 connect the seven localizers' recommendations to measured placement curves; Section 4.3 shows how the layer at which activations are captured changes recommendations; Section 4.6 separates repeatability from placement accuracy; and Section 5 / Table 3 examine whether a depth prescription transfers across models. We have clarified the methods and brought the two new studies into the main text.

When task-calibrated selection helps (Section 4.9, Table 2, Appendix C). We test Adaptive-LoRA's selection signal (Zuo et al., 2026), which was designed for adaptation: temporarily zeroing layer outputs measures how much each layer affects the model's predictions. On Qwen2.5-1.5B, we train eight fixed subsets of eight layers (called masks in the paper) with three seeds in each of two domains, using the same parameter budget and target-domain training data across subsets. The 48 fits are evaluated on the full PIQA, ARC-Challenge, GSM8K and SVAMP files (5,329 prompts).

PIQA improves by 2.38 percentage points over the five-random-subset mean (95% item confidence interval [1.23, 3.50], Holm-adjusted p = 0.000122 across four primary tests). The difference is positive in all three seed averages. The interval describes uncertainty over benchmark items for these fixed seeds and subsets. The margin over the best seed-averaged random subset is 0.15 points descriptively; the supported comparison is with the random mean. The other three primary comparisons remain unresolved; their intervals include both modest benefit and harm.

Target-domain calibration also beats selection calibrated on the other domain: PIQA +3.57 points and ARC-Challenge +2.87 points, with adjusted p = 0.0000327 and 0.0311. PIQA also beats pooled-common selection, which combines calibration prompts from both domains, by +1.78 points (adjusted p = 0.0259). These are a separate family of eight matching tests; the other five remain unresolved. ARC-Challenge thus shows a benefit from matching the calibration domain without establishing an advantage over random placement.

Complete coverage and the requested RelP test (Section 4.8, Figure 2, Appendix B). We train every contiguous nine-layer window in Gemma-2 9B under one protocol: continuation of the same initial adapter for 13 updates on 207 records, evaluated on 40 held-out tasks from eight JSON constructions. Accuracy ranges from 15/40 to 27/40. The best observed window, layers 24-32, was absent from the original grid. This directly tests the coverage concern, without treating the original seven centers as a global optimum.

Attribution patching (AtP), RelP and exact activation patching use the same ten development pairs, prediction target and attention-output locations across all 42 layers. AtP and exact patching choose layers 20-28 (22/40); RelP chooses 33-41 (16/40). RelP minus AtP is -15.0 percentage points (task-level 95% interval [-25.0, -5.0], exact McNemar p = 0.0703). Grouping uncertainty by construction gives [-35.0, +2.5], p = 0.375, so the evidence does not establish superiority. Descriptively, AtP is 9.41 points above the mean across all 34 windows; the interval across construction groups includes zero, so this does not establish an advantage.

Equal-parameter controls spread updates across all layers, use evenly spaced layers or use a scattered subset; they score 18/40, 17/40 and 18/40 and are not separated from either selector after correction. All 1,520 outputs reach the 1,024-token limit and remain counted; grading checks the first JSON object.

Scope of the original benchmark. The original code, JSON, regex and QA grids span Gemma-2 9B, Qwen-2.5 7B and Llama-3.1 8B. They test seven window centers plus three scattered random subsets at the corresponding budget; QA is descriptive, not graded. We disclose exact training/evaluation overlap (code 115/300, JSON 97/250, QA 20/183) and repeated evaluation records (214 distinct code pairs, 247 JSON pairs). These are comparisons on the recorded examples, not independent held-out generalization estimates. The revision tests localization as a placement tool without claiming universal success or failure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.