- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Programmable Semantic Interventions (PSI), which replaces the executable high-level causal model and counterfactual labels used in prior causal abstraction methods with a weaker “semantic program”: an outcome library and a transport rule over model-derived output distributions. PSI learns a low-rank activation subspace whose interchange intervention follows this declared transport rule, and further uses the same semantic coordinate system for donor-free control. Experiments on MIB tasks show that PSI is generally competitive with DAS, while requiring less supervision. An ablation further suggests that curated base-source pairing contributes little compared with having access to a larger donor pool.
Critical strengths
- The semantic-program interface is a useful simplification.Requiring only an outcome library and transport rule is substantially lighter than specifying a full executable high-level causal model and per-pair counterfactual targets.
- The supervision ablation is useful.The result that larger donor coverage can recover most of the fully supervised performance suggests that some of the supervision assumed by prior methods may be unnecessary.
Critical weaknesses
-
It is unclear whether the weaker specification also weakens the explanatory claim. By replacing a full high-level causal model with an output-level transport rule, PSI may move from identifying a variable within a high-level causal computation toward identifying a subspace that is simply effective at controlling the declared outputs. Since training and evaluation are both defined largely in terms of whether the output follows the declared transport, it is difficult to distinguish a true representation of the intended causal variable from a downstream output-control subspace.
-
The evaluation mainly uses variables with small and explicit outcome spaces. MCQA answer position, carry, and entity attributes are cases where the semantic program is relatively easy to specify. It is therefore unclear how much specification burden is actually saved for more abstract variables, where defining a clean outcome library and transport rule may itself be difficult.
Questions for authors
-
What stronger explanatory claim does PSI certify beyond finding a low-rank subspace whose intervention satisfies the declared output transport rule? In particular, how should PSI be distinguished from a semantically conditioned activation-steering method once the full high-level causal model is removed?
-
Can the learned carrier transfer across different realizations of the same semantic variable without retraining? For example, could an MCQA position carrier learned with A/B/C/D still work with 1/2/3/4 or another answer encoding? Such a result would make the causal-variable interpretation substantially more convincing.
-
How much of PSI’s advantage over DAS remains when both methods are restricted to exactly the same training observations and donor pool? The ablation suggests that donor-pool size explains a substantial part of the gain. A controlled comparison would help separate the benefit of the semantic-program formulation from the benefit of having access to more donors.
3 comments
Q3: How much of the advantage over DAS remains with the same observations and donor pool?
The reviewer's hypothesis is right. We retrained the official DAS code with the same training examples and number of updates as PSI (Gemma-2 2B, MCQA, layer 17, rank 16, 768 updates, last checkpoint, three seeds). DAS reaches 91.70 ± 0.93 and PSI 90.24 ± 0.52 (92.00 ± 1.40 with the reference variant of the loss). With 122 test pairs these differences amount to one or two pairs, so at matched data and budget the two perform on par. The margins in Table 1 therefore are mainly driven from the increase data availability, and we have adapted the language in our manuscript to reflect this fact (§3.1, Appendix A.8). Importantly, PSI reaches this parity with random pairs, and without any gold label when the correctness filter is removed (−0.9 points; Appendix A.7).
The benefit of the formulation itself can be separated from the donor pool by holding prompts and pairs fixed and changing only the training target:
| Training target | MCQA: MIB score | MCQA: KL to donor ↓ | ARC: KL to donor ↓ | ARC: same answer as a wrong donor |
|---|---|---|---|---|
| model's distribution (PSI) | 90.24 ± 0.52 | 0.103 | 0.020 | 85.9 ± 1.0% |
| one-hot on the model's top option | 88.55 ± 0.49 | 0.277 | 0.117 | 79.3 ± 3.9% |
| one-hot gold label | as above* | as above* | 0.090 | 72.1 ± 5.6% |
*On MCQA's filtered pool the gold label and the model's top option coincide. ARC: 2241 unfiltered natural questions, 589 held-out pairs, 111 of them with a donor the model answers wrongly (§3.2, Fig. 3).
With the donors fixed, reading the target from the model brings the edited model closer to the donor on both tasks. On ARC, where the model answers about a fifth of the questions wrongly, the subspace trained on gold labels transfers a wrong donor's answer in 72.1% of pairs, against 85.9% for PSI.
W2: Small, explicit outcome spaces
The answer list describes the outputs of the task, and the computation behind them never enters it, so how hard it is to write depends on the task's answer space more than on how abstract the variable is. The natural ARC questions of the new §3.2 show this: an executable high-level model for them would have to answer science questions, while the answer list is four symbols, and PSI localizes the variable there without any gold label. Many interpretability targets share this shape, such as multiple-choice, classification, yes/no questions and factual recall. Where the outputs are open-ended, naming a clean answer list is harder, and the savings depend on the variable, as we state in the Limitations (§4).
We hope that the transfer test the reviewer proposed, which shows that the subspace carries the position independently of the donor's symbols, together with the comparison of targets at fixed donors, addresses the concern about the explanatory claim and gives reason to reconsider the score. All changes are marked in blue in the revised manuscript.
Thanks for the detailed rebuttal. The additional experiments address most of my concerns, particularly the cross-encoding transfer and fair comparison with DAS. While the causal interpretation and generalizability remain somewhat limited, I appreciate the contribution of achieving comparable performance with weaker supervision. I have raised my score from 2 to 3.
We thank the reviewer for the careful review and for three precise questions, each of which we ran as an experiment. The test the reviewer proposed in Q2, transfer across answer encodings, also gives the most direct answer to the main concern (W1), so we discuss the two together.
W1, Q1 and Q2: A causal variable, or a subspace that controls the output?
Q2 gives a direct way to tell the two apart. A subspace that controls the declared outputs would write the donor's answer, while a subspace that carries the variable writes the donor's position in the recipient's own symbols. We froze the MCQA subspaces (Gemma-2 2B, layer 17, rank 16), trained with the benchmark's uppercase option labels, and re-encoded the options of donor and recipient independently as lowercase letters, digits or per-prompt permuted digits, with no retraining (16 combinations, 150 pairs each; §3.1, Appendix A.10, Fig. 6).
The learned subspace writes the position. With an uppercase recipient, it transfers the position from lowercase donors in 100% of pairs and from digit-labelled donors in 92.4%, at least as well as between two prompts with the original labels (86.2%). It never outputs the donor's own symbol: with a lowercase donor and an uppercase recipient, this happens in 0 of 150 pairs. In the 44 test pairs where both the position and the symbols change, the edited model gives the recipient's symbol at the donor's position in 66.7% of pairs, and the donor's literal symbol in 9.1%. Copying the full activation at the same site does carry the token (with a letter donor and a digit recipient, it outputs the donor's letter in 44% of pairs), so the subspace selects the position from an activation that also holds the answer.
The one encoding that does not transfer well is a digit-labelled recipient: there the edit moves the answer in 45–49% of pairs, against 32% with no edit, although the model answers digit-labelled questions correctly 95.0% of the time. Our reading is that digit-labelled prompts do not rely on this subspace alone to produce their answer. DAS and one-hot targets at the same budget show the same pattern, so it belongs to the model at this site, and all the methods we measured agree on it.
On the explanatory claim, removing the executable high-level model does not change what is certified about the tested variable. Interchange accuracy scores a high-level model only through its outputs on the tested pairs, so for the variable under test PSI and DAS certify the same kind of claim, and the transfer results above hold for both. What a full high-level model adds is hypotheses about other variables, and in PSI each of these can be tested with its own program.
On the difference from steering: in an interchange the experimenter chooses nothing, since the value written is whatever the donor prompt's activation holds, and a subspace that could only push the output in a fixed direction would fail the test. The controller of §2.3 is steering by design. What the paper adds is where it writes: inside a subspace that has passed the interchange test, in coordinates the program defines. This matters in practice: the same controller realizes 94.8% of held-out requests when trained jointly with PSI's subspace, against 83.9% in the top principal subspace of the same rank and 34.2% in a random one (§3.3, Fig. 4).