acceptodds
Under review as a conference paper at ICLR 2027

From Similar Code to Neural Correspondence: Causal Interventions Guided by Program Structure

Abstract

Large language models can generate code that closely resembles existing implementations, but output similarity alone does not explain how similar code is supported by the model's internal computation. We study whether program structure can help identify internal states that play corresponding causal roles across similar programs. We introduce Program Structured Causal Alignment (PSCA), a framework that uses program structure to select corresponding positions across programs and evaluates the transferred internal states through causal intervention. PSCA evaluates this correspondence in two complementary ways. Intact causal compatibility asks whether transfers selected by program structure better support code prediction than comparison transfers in an otherwise intact computation. Causal recovery asks whether they restore part of the lost prediction after disruption and achieve greater recovery than comparison transfers. During intact computation, transfers selected by program structure generally support prediction better than comparison transfers. Across three model families and nine fine-tuned models, data-flow-aligned restoration consistently improves prediction over both the disrupted condition and mismatch restoration at the prespecified intervention layers, while control-flow recovery varies across models and intervention settings. Together, these results provide evidence that program structure can guide the identification of internal states with meaningful functional effects on code prediction.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Activation Patching, Program Structure, Code Generation, Large Language Models, Causal Intervention

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_arPaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper asks whether program structure can identify internal states with corresponding roles across similar programs. PSCA extracts CFG/DFG relations, projects endpoints to token spans, and transfers complete hidden vectors through activation patching. Structure-aligned transfers are compared with same-offset, constrained-random, and clone-span transfers. The evaluation covers intact compatibility, post-disruption recovery, and restoration specificity across nine fine-tuned models from three model families.

Critical strengths
  • The question is clear and productively separates output-level code similarity from internal computation.
  • The empirical effort is substantial: CFG and DFG alignments, full layer sweeps, separate compatibility and recovery protocols, and within-pair controls. The authors also acknowledge that complete hidden states do not isolate role, lexical, or positional information.
Critical weaknesses
  • The main effect is not isolated from surface confounds. Complete hidden vectors mix lexical, positional, and program-role information. Appendix F.3 gives an example in which a purely lexical coordinate can produce a positive compatibility effect. The paper includes clone-span and matched-generic controls, but these do not residualize lexical/positional subspaces or provide an exact nuisance-matched donor. The results support functional compatibility more strongly than correspondence of program-role representations.
  • Training/evaluation independence is a central concern. Appendix A reports 3,030 training source functions and 3,030 selected greedy query records per model, 2,813 query families, and a library containing the originals of all 3,030 training functions. This strongly suggests substantial prompt/source reuse, and no disjoint held-out split is reported. Exact overlap statistics and a held-out rerun are needed before the effects can be interpreted as general correspondence rather than familiarity.
  • Statistical dependence and the strongest baseline need more attention. The nine models share source prompts and control examples. Main effect intervals are pair-bootstrap, not query/source-family clustered; Appendix C's family resampling concerns the NLL/KL correlation rather than the primary intervals. C.8 evaluates a clean-state neural cosine selector on the same subsets as the same-offset/constrained-random comparisons (1,279 CFG and 3,621 DFG), not the full main-effect populations. Its NLL intervals are below zero in five of eighteen model-by-abstraction comparisons (three CFG and two DFG; three of nine configurations) and include zero in the other thirteen; KL is not similarly one-sided. The selector also reproduces PSCA's assignment in most cases, so it belongs in the main table with clustered uncertainty.
  • Novelty needs sharper empirical positioning. The contribution is the integration of relation matching with activation patching. The closest code-specific work, Abdelsalam et al. (2026), already applies activation patching to programs, and alignment-learning methods such as DAS/HyperDAS already provide alternatives. The paper explains its distinction, but it does not empirically test what program-structure-derived alignment adds beyond these approaches.
Questions for authors
  1. What are the exact function- and prompt-level overlap counts between training and evaluation, and do the conclusions hold on a fully disjoint held-out set?
  2. Can the authors add lexical/positional residualization or a matched-donor control that changes structural correspondence while holding surface features fixed?
  3. Can the neural cosine comparator be included in the main comparison with query/source-family clustered confidence intervals?
  4. Can the authors describe the model in detail, the model size and series for easier reproduction. Qwen is not informative.
3 comments
AuthorsOpenReview ↗

Reply to Reviewer arPa (1/2)

W1/Q2. Surface confounds and a matched-source control.

Answer: We added a matched-source control (your “matched donor”) to the structural binding benchmark. When both source programs use the same name at the final use, changing the binding shifts the target prediction further toward the alternative type than preserving it.

In the first source program, the renamed variable refers to the same assignment as in the target program. In the second source program, the same name refers to a different assignment. We transfer the state from each source program’s final variable use to the same position in the target program. We also keep the prediction position the same. The table shows how much more a binding change reduces the logit difference (LD) between the model’s scores for method names of the target and alternative types. Each row averages the Base, Original, Lexical and Structural models of one architecture (Table 29).

ArchitectureDifference in LD reduction
Gemma3.33
Llama1.73
Qwen4.73

We also compared binding changes with name changes. The median LD reduction was 2.49 when we changed the binding, 0.05 when we renamed the queried variable, and below 0.01 when we changed only the names in the other two definitions. On the retrieved-program benchmark, we measure complete-state compatibility and restoration. [§3.5, Figure 5 and Appendices F.3, G.1]

Q1/W2. Training overlap in the main and held-out sets.

Answer: We generated each target program in the main set from the task instruction of a function used in fine-tuning (the target’s origin function). Both the function and its instruction appeared in fine-tuning for all 819 CFG and 2,731 DFG query families (each origin counted once per configuration). The reformatted inference prompts had zero exact matches to the training instructions. To retrieve source programs, we built a database of 51,191 distinct Python functions from 120 GitHub repositories, including the originals of all 3,030 source functions used in fine-tuning. The retrieved source appeared in fine-tuning for 1,371 of 1,566 CFG pairs and 3,519 of 3,813 DFG pairs.

Retrieved sourceCFG pairsDFG pairs
The target's origin function758 (48.4%)2,601 (68.2%)
Another fine-tuning function613 (39.1%)918 (24.1%)
A function outside fine-tuning195 (12.5%)294 (7.7%)

For the held-out set, we excluded fine-tuning functions and their close matches before selecting origin functions and retrieving source programs. The held-out target instructions had zero exact matches to the fine-tuning instructions. DFG compatibility and restoration specificity were positive for the Original and Base models of Gemma, Llama and Qwen. CFG restoration specificity was positive in three of these six models. In the main set, restoration specificity was positive in 9/9 DFG and 8/9 CFG configurations both for transfers from the target program’s origin function and for transfers from another function (non-origin). The table gives mean DFG advantages over mismatch transfer (NLL, nats/token). [§3.1, §3.4 and Appendices A.2–A.3, C.9–C.11]

SettingCompatibilityRestoration specificity
Held-out set: Original / Base models (Table 24)0.28 / 0.260.34 / 0.33
Main set: Original / Base models (Table 25)0.22 / 0.170.38 / 0.28
Main set: origin / non-origin sources (Table 23)0.21 / 0.170.34 / 0.33

W3/Q3 (A). Statistical dependence.

Answer: Program pairs can share a target origin function or a retrieved source function. To account for this dependence, we estimated 95% confidence intervals by resampling groups of pairs. We grouped pairs by target origin in one analysis and by retrieved source in the other. With either grouping, DFG recovery and restoration specificity were significant in all nine configurations for both NLL and KL.

Counting each group once per configuration gives 819 query families and 973 source groups for CFG, and 2,731 query families and 2,870 source groups for DFG. We averaged cases within pairs and gave each pair equal weight. We tested each configuration separately and report the mean effects across all nine (NLL / KL, nats/token).

EffectCFG (NLL / KL)DFG (NLL / KL)
Compatibility0.0138 / 0.01970.1997 / 0.2426
Recovery0.0245 / 0.03130.4954 / 0.5160
Restoration specificity0.0132 / 0.01850.3406 / 0.3808

With either grouping, DFG compatibility was significant in eight of nine configurations for NLL and all nine for KL. For CFG recovery, the NLL effect was significant in three of nine configurations when grouped by target origin and four when grouped by source function. For KL, it was significant in seven of nine with either grouping. [§3.1, Appendix C, Table 7]

AuthorsOpenReview ↗

Reply to Reviewer arPa (2/2)

W3/Q3 (B). Cosine baseline.

Answer: We added the Cosine baseline to the main comparison and report two sets of 95% confidence intervals, one clustered by target origin function and the other by retrieved source function. Given the same target positions, PSCA selects source positions from program structure alone (black-box selection) and Cosine uses activations from both programs (white-box selection). We compared the methods on 1,404 of 1,566 CFG pairs and 3,746 of 3,813 DFG pairs. During intact transfer, PSCA and Cosine selected exactly the same positions in 60.9–89.8% of DFG cases and 88.6–95.6% of CFG cases across the nine configurations. For each architecture, the table gives mean NLL and KL (nats/token) across the Original, Lexical and Structural configurations during intact DFG transfer.

NLL ↓ / KL ↓PSCACosineSame-offsetRandom
Gemma0.1802 / 0.12910.1708 / 0.11970.7522 / 0.70600.7296 / 0.6822
Llama0.5467 / 0.01810.5459 / 0.01761.1201 / 0.65281.1244 / 0.6662
Qwen0.5189 / 0.04500.5070 / 0.03211.1847 / 0.77881.1584 / 0.7579

Across all three architectures, Cosine gave slightly lower mean NLL and KL than PSCA. Lower NLL indicates better prediction of the target tokens, and lower KL indicates closer agreement with the clean target distribution. We also compared the methods within each configuration for CFG and DFG (18 comparisons per metric). With either grouping, most comparisons showed no clear difference between PSCA and Cosine during intact transfer. Where the difference was clear, NLL or KL was lower with Cosine. [§3.2, Table 1 and Appendices C.6–C.8]

Q4. Model series and sizes.

Answer: We use Gemma-2-9B-it, Llama-3.1-8B-Instruct and Qwen2.5-Coder-7B-Instruct. Base denotes the released instruction-tuned model. Original denotes the same model after we fine-tuned it on original code. Appendix A gives our fine-tuning settings.

W4. Contribution of program structure to state transfer.

Answer: Across all nine DFG configurations, NLL was 0.42–0.71 nats/token lower with PSCA than with same-offset or random selection. Abdelsalam et al. (2026) apply activation patching to program variants whose token positions are aligned by construction. PSCA identifies corresponding token positions by matching control-flow or data-flow relations between the target program and a retrieved source program. We kept the target positions the same across the three selection methods to measure what program structure contributes to source selection. [§3.2, Table 1]

On the structural binding benchmark, distributed alignment search (DAS) distinguished changes in binding more clearly at PSCA positions than at mismatch positions. PSCA selects the source positions, and DAS learns which subspace of the state to transfer. HyperDAS also learns which positions to use. We trained DAS separately at PSCA and mismatch positions. At both sets of positions, we measured the logit difference (LD) between scores for method names of the target and alternative types. The table shows how much more a transfer reduces LD when the source program changes the binding than when it preserves it. Each row averages the Base, Original, Lexical and Structural models of one architecture, using the same cases within each model for both columns (Table 35).

ArchitectureDAS at PSCA positionsDAS at mismatch positions
Gemma5.430.003
Llama3.35−0.004
Qwen5.910.000

At PSCA positions, changing the binding shifted the prediction further toward the alternative type in all three architectures. At mismatch positions, both transfers shifted the prediction, but the difference between the two source programs was close to zero. [§3.5 and Appendix G.4]

Dear Reviewer arPa,

Thank you again for your thoughtful review and constructive feedback on our submission.

As the reviewer–author discussion period approaches its conclusion, we would be grateful if you could revisit our responses. We hope our clarifications and additional experimental results have helped address your concerns, and we would be happy to provide further clarification or respond to any additional questions you may have.

Thank you very much for your time and consideration.

Best regards,

Author(s)

Reviewer_qGAZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether program structure can identify corresponding neural states that play similar causal roles across related programs. The authors propose Program Structured Causal Alignment (PSCA), which constructs token-level alignments using control-flow and data-flow correspondences and evaluates them through activation patching. The authors investigate two settings: 1. intact causal compatibility, which measures whether structure-aligned state transfers better preserve code prediction than mismatched transfers, and 2. causal recovery, which measures whether such transfers can restore prediction after disrupting the target computation. Experiments across three model families and nine fine-tuned models show that structure-aligned transfers generally outperform mismatched transfers. The results are particularly consistent for data-flow alignment, while control-flow alignment exhibits smaller and more variable effects.

Critical strengths

The paper studies an interesting question: whether correspondences defined independently of neural activations predict the functional compatibility of internal states across programs. This helps us understand the mechanism of code LLMs.

Critical weaknesses

Potential gap between motivation and evaluation. The introduction motivates the study by asking whether structurally corresponding program elements play corresponding causal roles in code generation. However, the evaluation relies solely on NLL and KL divergence. While these metrics quantify causal effects on predictive distributions, they do not establish whether the interventions meaningfully affect the final generated programs. Specifically, two predictive distributions can differ while producing the same output under decoding.

Fine-tuning effects. Because the central question concerns structural correspondence in model representations, it is unclear why the evaluation is limited to fine-tuned checkpoints. Fine-tuning may create, strengthen, or reorganize the very correspondences being measured. The author should either motivate why the fine-tuning is necessary or compare the fine-tuned models with corresponding base models under the same setting.

Weak structural controls. The evaluation appears to retrieve related source–target programs using JPlag, identify corresponding locations through structural matching, and compare transfers between those locations with deliberately mismatched assignments. Under this construction, aligned positions may share lexical content, token type, syntactic category, relative position, local context, and graph role simultaneously. Consequently, the aligned v.s. mismatched comparison may not isolate program structure. Although successful activation transfer is not guaranteed only because two programs are similar, the current mismatched control may be too weak to support a strong claim about abstract structural representations.

Questions for authors

See weakness.

  1. The position list appears to be constructed based on CFG and DFG only. CFG and DFG matching identifies structural relationships, but it does not by itself establish that two code regions perform the same function. For example, two loops can have similar control-flow roles while implementing unrelated computations. Does the dataset treat such regions as aligned? If so, were their intervention effects different from those of functionally similar aligned positions or structurally misaligned positions?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer qGAZ

W1. Effects on decoded code.

Answer: We added exact match (EM) to test whether aligned transfer helps the model reproduce the target continuation more often than mismatch transfer. Aligned DFG transfer reproduced the scored continuation more often than mismatch transfer in all nine configurations.

A case counts as an exact match if, given the preceding target tokens, the model ranks the target token first at every scored position. Greedy decoding from the same prefix with the same intervention then reproduces the scored continuation.

For each architecture, we average intact DFG results across the Original, Lexical and Structural configurations.

ArchitectureClean EM (%) ↑Aligned EM (%) ↑Mismatch EM (%) ↑
Gemma79.175.360.7
Llama35.332.221.8
Qwen45.341.827.7

Table 26 reports a DFG advantage of 5.5–22.4 percentage points during intact computation and 2.8–19.6 points after restoration. For CFG, aligned transfer gives higher EM in eight of nine configurations during intact computation and seven after restoration. [§3.3–3.4; Appendix C.12]

W2. Effects before and after fine-tuning.

Answer: DFG compatibility and restoration specificity are positive in all three Base models before our fine-tuning. We compared each Base model with its Original model, fine-tuned on original code. We used the same programs, intervention positions and scored continuations for both models (Table 25).

Entries give the NLL advantage of aligned over mismatch transfer (nats/token) for Base / Original. Larger values indicate a greater advantage.

EffectGemmaLlamaQwen
Compatibility0.085 / 0.1800.209 / 0.3240.207 / 0.170
Restoration specificity0.225 / 0.3860.310 / 0.4530.296 / 0.295

Both effects are larger in the Original models for Gemma and Llama. For Qwen, the compatibility estimate is lower in the Original model and restoration specificity is nearly unchanged. [Appendix C.11]

W3. Structural, lexical, contextual and positional controls.

Answer: We added a structural binding benchmark to test whether state transfer depends on which assignment supplies the value at the final use. Changing that binding shifts the target prediction more than preserving it, even when both sources use the same variable name.

In the source, we change that binding or the assigned value (structural), rename the queried variable throughout the program (lexical), or change only the names in the other two definitions (contextual). We transfer the state from each source’s final use into the same target, keeping the transfer and prediction positions the same across conditions.

The type logit difference (LD) measures how much higher the score is for method names of the target type than for those of an alternative type. A larger LD reduction relative to the clean target indicates a stronger shift toward the alternative type. The table gives the median of the twelve model means (Base, Original, Lexical and Structural for each architecture). The assigned-value row uses cases where the values can be swapped.

FactorSource changeMedian LD reduction
StructuralBinding2.49
StructuralAssigned value1.56
LexicalVariable name0.05
ContextualNames in the other two definitions<0.01

Table 29 reports the same-name comparison. Across the twelve models, LD reduction is 1.71–4.97 larger for the source that changes the binding. [§3.5, Figure 5; Appendix G.1–G.2]

Q1. Structurally aligned regions that perform different computations.

Answer: Yes. On the structural binding benchmark, PSCA aligns the final use in both sources with the same target use even when the assigned values differ in type. PSCA selects positions by their CFG or DFG relations.

We keep the binding the same in both sources. One source rewrites the assignment while preserving the target value, for example by replacing Path(".") with Path(). The other assigns a value of a different type. We transfer from the aligned use in both sources and from a mismatch position in the value-preserving source when available.

Table 33 reports LD reductions for each model. For the table below, we averaged the four models within each architecture.

ArchitectureAligned, value preservedAligned, value type changedMismatch, value preserved
Gemma−0.0210.810.70
Llama−0.0132.081.20
Qwen−0.0150.543.33

At the aligned use, the value-preserving rewrite barely changes the gap between the two type scores. Assigning a value of a different type shifts the scores further toward that type in all twelve models (difference in LD reduction, 0.48–2.20). For the value-preserving source, mean LD reduction at mismatch positions is positive in each architecture. [Appendix G.3]

Dear Reviewer qGAZ,

Thank you again for your thoughtful review and constructive feedback on our submission.

As the reviewer–author discussion period approaches its conclusion, we would be grateful if you could revisit our responses. We hope our clarifications and additional experimental results have helped address your concerns, and we would be happy to provide further clarification or respond to any additional questions you may have.

Thank you very much for your time and consideration.

Best regards,

Author(s)

Reviewer_ETumOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Program Structured Causal Alignment uses control-flow and data-flow relations to select corresponding token positions across similar programs before evaluating neural interventions. Hidden states are transferred between programs to test compatibility during intact computation and recovery after an earlier disruption. Experiments across three model families and three fine-tuning settings compare structural alignment with matched mismatches and alternative alignment rules. DFG transfers show consistent benefits in continuation likelihood and divergence from clean predictions; CFG effects depend more on the model and intervention setting.

Critical strengths

correspondences before neural intervention outcomes are observed. The mismatch controls retain target positions and write budgets while changing source correspondence. This makes the comparisons informative about the proposed alignment and reduces straightforward explanations based on where or how much of the computation is modified. 2. The paper distinguishes preserving an intact computation from recovering a disrupted one, and Appendix F formalizes why the former need not imply the latter. Testing both properties makes the conclusions easier to interpret and avoids treating a relatively harmless patch as sufficient evidence of functional replacement. 3. At the reported restoration settings, aligned DFG transfers improve NLL over mismatched transfers by approximately 0.24–0.44 nats/token across all nine model and training combinations. KL results agree with this pattern. The consistency across architectures and training transformations gives the main empirical finding more weight than an isolated successful intervention.

Critical weaknesses
  1. The interventions establish correspondence between complete states, but do not isolate the contribution of program-role information. Lexical, positional, and contextual information can also affect transfer success. Appendix F.3 acknowledges this and gives a lexical counterexample under matched write geometry. Consequently, the results support the usefulness of structurally selected transfers more directly than a claim that the model represents the corresponding program roles in a causally specific way.
  2. The study uses a deliberately constructed population of familiar functions. Appendix A describes 3,030 source-function identities used in variant fine-tuning and query construction, with their original versions included in the retrieval library. Retrieval and projection filters then determine the intervention cohorts. This is a reasonable controlled setting, but it does not establish that similar correspondence will occur for unfamiliar generated programs. The external reach of the finding should remain limited to the tested population.
  3. The coverage of the primary DFG layer analysis needs clarification. Section 3.1 states that full layer sweeps were evaluated, whereas Appendices A and C.3 describe Population A results at fixed compatibility and restoration anchors. The serialization study in Appendix E uses a different historical population. It is therefore difficult to assess how sensitive the main DFG result is to these choices. Identifying and reporting the Population A sweeps would address this concern.
Questions for authors
  1. Can structural-role agreement be varied independently of lexical or contextual agreement while retaining the same intervention geometry?
  2. How often is the donor the query’s registered training source rather than another retrieved function in each primary cohort?
  3. Which full layer sweeps in Section 3.1 cover DFG Population A, and where are their compatibility and recovery results reported?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer ETum

W1/Q1. Structural, lexical, positional and contextual controls.

Answer: We added a structural binding benchmark to examine structural, lexical and contextual factors separately. We transfer the state from each source program’s final variable use into the same target program and keep the transfer and prediction positions the same across conditions. Even when both source programs use the same variable name, changing the binding shifts the target prediction further toward the alternative type than preserving the binding.

Consider a target program with the assignments k = [*items], s = str(text), and h = open(path, mode="r"). At the final use, we compare the model’s scores for methods of the target type and an alternative type. In this example, the target type is list and the alternative type is str.

For each case, the type logit difference (LD) is the score for the target type minus the score for the alternative type. A larger reduction in LD after transfer indicates a stronger shift toward the alternative type. The table illustrates the edits and reports median LD reductions across twelve models (Base, Original, Lexical and Structural for each of three architectures).

FactorSource editSource use in this exampleMedian LD reduction
Structural: bindingSwap definition names k and sk refers to the string2.49
Structural: valueSwap assigned valuesk refers to the string1.56
Structural: jointSwap names and valuesk refers to the list again0.28
LexicalRename k to dd refers to the list0.05
ContextualSwap or rename s and hk still refers to the list<0.01

We also compare two sources with the same new variable name: one preserves the binding and the other changes it. Table 29 shows a larger LD reduction for the source that changes the binding in all twelve models, with differences ranging from 1.71 to 4.97. On the retrieved-program benchmark, we measure complete-state compatibility and restoration. [§3.5, Figure 5; Appendix G.1–G.2]

Q2/W2. Source origin and evaluation scope.

Answer: The main retrieved-program analysis covers nine fine-tuned configurations (three architectures × three training sets). Each target is generated from the task instruction of a fine-tuning function. We call this function the target's origin function (your "registered training source"). The retrieved source is this origin function in 48.4% of CFG pairs and 68.2% of DFG pairs overall. Table 4 reports the percentages by configuration, in Original/Lexical/Structural order.

ArchitectureCFG (%)DFG (%)
Gemma52.8 / 42.4 / 51.069.1 / 59.7 / 60.8
Llama53.2 / 45.7 / 51.178.8 / 75.7 / 73.5
Qwen39.9 / 37.1 / 45.171.0 / 61.2 / 67.4

For both origin and non-origin sources, DFG restoration specificity is positive in all nine configurations (Table 23).

SourceRestoration specificityPositive
Origin0.320 (0.267–0.473)9/9
Non-origin0.363 (0.129–0.533)9/9

Entries give the median and range over the nine configurations, in NLL (nats/token).

  • Held-out functions. Target-origin and source functions are disjoint from our fine-tuning functions. DFG restoration specificity is positive in the Original and Base models of all three architectures (Table 24).
  • Base models, before our fine-tuning. DFG compatibility and restoration specificity are positive in all three Base models (Table 25).

The DFG advantage is therefore present with non-origin sources, on the held-out set, and before our fine-tuning. [Appendix A.2–A.3, C.9–C.11]

Q3/W3. DFG layer sensitivity and coverage.

Answer: We added a layer-by-layer analysis of the main DFG cases from the retrieved-program benchmark (§3.1). In each of the three training settings, aligned transfer yielded lower mean NLL than mismatch transfer at all 42 layers in Gemma, all 32 layers in Llama, and all 28 layers in Qwen.

The analysis covers 3,813 source–target program pairs and 42,227 cases across nine configurations. We also tested restoration 1, 5 or 10 layers after disruption. At every layer pair examined in each configuration, aligned restoration yielded lower mean NLL than both leaving the target disrupted and restoring it with a mismatched state. Figures 8–9 in Appendix C.1 report the full results.

Dear Reviewer ETum,

Thank you again for your thoughtful review and constructive feedback on our submission.

As the reviewer–author discussion period approaches its conclusion, we would be grateful if you could revisit our responses. We hope our clarifications and additional experimental results have helped address your concerns, and we would be happy to provide further clarification or respond to any additional questions you may have.

Thank you very much for your time and consideration.

Best regards,

Author(s)

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank all reviewers for their careful reading and constructive feedback. Their comments motivated additional experiments and analyses and helped us clarify the experimental design and the paper's contribution. PSCA uses program structure to identify corresponding positions across programs before examining model activations, then tests whether states from those positions support target prediction better than states from mismatched positions in the same source program. We summarize the main revisions below.

1. Structural, lexical, contextual, and positional controls. We constructed the structural binding benchmark to test how edits to a source program affect predictions in the same target program after state transfer. We ask whether changing the queried variable’s binding or assigned value in the source (structural) shifts the target’s type preference more than renaming that variable (lexical) or changing only the names of other variables (contextual). We keep the transfer and prediction positions unchanged across these comparisons (positional). [§3.5, Figure 5; Appendix G.1–G.3]

2. Comparison with alternative alignments. We included the Cosine baseline in the main comparison to compare PSCA’s black-box selection of source positions with white-box selection based on activation similarity. We trained distributed alignment search (DAS) separately at PSCA and mismatch positions to test whether changing the binding in the source program has a stronger effect on the target prediction at PSCA positions. [§3.2, Table 1; Appendix C.6–C.8; §3.5, Appendix G.4]

3. Broader evaluation. We added the exact match (EM) metric to evaluate output-level effects beyond NLL and KL. We expanded the layer analyses to examine how transfer effects vary with depth. We report clustered confidence intervals and compare transfer effects for origin and non-origin sources. To examine dependence on fine-tuning, we evaluated the Base models and tested transfer on functions outside our fine-tuning set. [§3.1, §3.3–3.4; Appendix A.3, C, C.1, C.9–C.12]

4. Clearer presentation of experimental design and findings. We reorganized the Experiments section to describe the setup in §3.1 and present each comparison alongside its main finding in §§3.2–3.5. [§3.1–3.5]

Revised text appears in pink. We address each reviewer’s questions and concerns in the individual responses below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.