acceptodds
Under review as a conference paper at ICLR 2027

Local Matching Can Miss Finite Behavior in Model-Edit Interventions

Abstract

Local measurements are often used to characterize interventions in a language model. When does agreement on these measurements imply similar effects at a finite intervention scale? We introduce \finitefiber, an audit that constructs distinct interventions with matched local profiles and then compares their complete-target log probabilities. The evaluated score is the same score whose derivatives enter the matching constraints. On an independent cohort of 40 Qwen2.5-3B factual edits, a profile matching residual geometry, positionwise derivatives at two anchors, calibration energies, and directional curvature admits controls for 35 facts. With a 1% energy tolerance, 23 facts exhibit score differences of at least nat/token; the prespecified majority-rate test is not significant (). A separate shared eight-dimensional activation family clarifies the role of measurement resolution: two aggregate slopes admit finite-effect differences, whereas retaining the positionwise readings of the same edited-anchor gradient identifies the linear coefficients exactly on all 40 facts. This stronger measurement also rejects controls with small score differences, so it does not establish a selective verification procedure. Local scores nevertheless rank restricted LoRA subedits effectively. These results distinguish local equivalence testing from candidate ranking and show that the implications of local agreement depend on the candidate family and the information retained by the measurement.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, selective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NktFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether matching local measurements is enough to guarantee similar effects when interventions are applied at a finite scale inside a language model.
The proposed FINITEFIBER audit constructs different activation interventions that match a specified set of local measurements within numerical tolerances. These measurements include residual norms, directional derivatives, calibration energies, and, in the stricter setting, a directional second-order term. The experiments show that matched interventions can still differ in their mean log probability per token for the complete target answer.
A separate experiment in a shared eight-dimensional intervention family shows that aggregating gradient measurements can lose information needed to distinguish candidates. Keeping positionwise measurements restores exact identifiability in this family.

Critical strengths

1.The paper goes beyond a general intuition about the limits of local measurements by directly constructing matched control interventions. It separates candidate construction, constraint verification, and finite-effect evaluation.
2.The experiments provide stronger evidence than a simple failure of first-order approximation. The stricter setting also matches a directional second-order term and uses tighter calibration-energy tolerances.
3.The shared-family experiment clearly demonstrates information loss from aggregation and the value of retaining positionwise measurements.

Critical weaknesses

1.The conceptual novelty beyond the known limits of local information in nonlinear functions is not sufficiently clear. Although the matched-control audit is carefully designed, the paper could better explain what new insight it provides beyond this general principle.
2.The interventions in Section 3.1 are deliberately constructed and substantially separated. The reference residual is amplified after projection, and the control is constrained to be orthogonal to it. These choices raise questions about how well the findings apply to realistic model edits, especially interventions implemented through shared weight changes.
3.The main confirmation under the strict matching profile is limited to Qwen2.5-3B at one layer and with one editing setup. Although the appendix includes experiments on other models, they do not directly replicate the main strict-profile result. Evidence for the generality of this result therefore remains limited.

Questions for authors

N/A

Reviewer_gEsuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether agreement on local measurements of activation interventions guarantees similar finite effects on a language model's target-answer likelihood. The authors construct pairs of interventions matched on local properties like residual norms, or endpoint directional derivatives.The authors find pairs with substantial finite-score differences in Qwen2.5-3B. The paper then studies linear identifiability.

Critical strengths

The paper has an interesting angle trying to connect local geometrical properties with identifiability problems.

Critical weaknesses

I am highly suspicious of significant AI writing, which tends to create these particularly difficult papers to read. Such papers read as reports from a coding agent to a user with a shared history of technical discussion rather than an effort of a researcher to take a step back once the work is finished to create a logical argument that progresses through clear motivations and intuition. The result is writing that jumps directly to weirdly specific technicalities, misses the general point, and no amount of post-hoc human editing will fix the structural flaw. Just to give one example out of many, line 45 weirdly introduces specific technical terms that are not motivated by the previous paragraph and are not necessary to motivate the next. Particularly, in line 47 we suddenly have "For a paraphrase p, [...]" — why "paraphrase" appears out of nowhere, why not an "input" or "prompt," is because the AI agent already knows it has to speak about paraphrases because much later in the paper we will discover the experiments are done on paraphrases. A paper can be technically dense, but it should still follow a meaningful logical progression.

The primary finding is that matching finitely many local derivatives and quadratic measurements does not guarantee agreement of a nonlinear function at finite displacements. This is mathematically unsurprising. In general, the paper's novelty (to the extent that I can understand it) appears insufficient.

Questions for authors
  • What is the primary methodological contribution of FINITEFIBER?
  • What do the findings say about real existing editing methods?
Reviewer_N7XcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a question: if two interventions on a model match the same local metrics profile, will they necessarily produce the same behavior? The authors frame this question as a sufficiency issue and propose an audit framework that first constructs alternative interventions matching a specified local profile, then compares their behavior with the reference intervention, and finally reports the differences. Experiments use a LoRA update on a single MLP down-projection matrix of Qwen2.5-3B, together with restricted interventions with a low degree of freedom. Results show that agreement on local metrics does not imply equivalent behavior, and the conclusion that can be inferred from the local agreement depends on the range of the candidates and the number of independent constraints provided by the measurement.

Critical strengths
  1. This paper designs an executable construct-verify-evaluate procedure that measures the behavioral gap between candidates matched under a given tolerance.
  2. This paper distinguishes local equivalence testing from candidate ranking as problems with different requirements, so the use of local metrics can be defined separately for each.
Critical weaknesses
  1. This paper only tests a LoRA update on a single parameter matrix and does not test any model-editing method from the existing literature. The candidates are defined in activation space and are not reproduced by actual weight updates; the authors themselves describe the result on actual weight updates only as a qualitative trend. Moreover, because the candidates have more degrees of freedom than the constraints provided by the local measurements, counterexamples must exist. The experimental conclusions therefore provide no novel interpretation.
  2. The core proposition refuted by this paper is defined by the authors themselves: agreement of local measurements within tolerance is sufficient to guarantee behavioral equivalence. The paper states that this proposition does not correspond to any published evaluator, so the negative result cannot be generalized. Experimental results show that among 40 edited facts, 23 exhibit a clear behavioral difference, but this proportion is not significantly above one half (p=0.215), the threshold set by the authors themselves.
Questions for authors
  1. For model-editing methods from the existing literature (e.g., ROME or MEMIT), can you provide candidates that are produced by actual weight updates, satisfy the paper's matching criteria, and exhibit a clear behavioral difference?
  2. Can you test the conclusion that "agreement of local measurements within tolerance does not guarantee behavioral equivalence" on any published evaluation practice and report the results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.