- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces PLOT, which is a gradient-free approach for discovering joint effects using optimal transport. They evaluate the approach on increasingly difficult tasks (hierarchical equality, binary addition, MCQA) and on increasingly complex models (an MLP, an RNN, and a Transformer). In general, PLOT is seen as a supplement to methods like Distributed Alignment Search (DAS), which tries to identify neural subspaces that implement one (or more) abstract variables. Assuming a correctly specified causal model, PLOT tries to discover its implementation in the neural network without using subspace analysis or gradients. My understanding of how PLOT works is:
- They require 3 datasets: one for fitting, one for calibration, and one for testing
- Measure abstraction intervention effects by swapping values (in the causal model) between two examples and recording the output change
- Measure neural intervention effects by swapping the base coordinates of a candidate neural site (for instance a PCA direction or a neuron) by the ones corresponding to the activations for the alternatively value in step 2
- Fitting an optimal transport coupling between the two effects
- Use the fitted coupling and the calibration dataset to calibrate intervention strengths (maximizing for largest logit agreement between intervented token and abstract target)
- Evaluating on a held out dataset.
PLOT additionally becomes hierarchical by considering its step 1 and 2 (2, 3, and 4 in my list above) applied first to coarser sites and then fitting new couplings within them. A concrete example from the paper is first considering token-wise plot, then layer-wise PLOT and then within-layer PLOT. This lets the method potentially be applicable also to large models by gradually honing on the correct neural sites that implement the abstract model.
They validate the method on 3 experimental configurations that increase in complexity:
- A ReLU MLP with 3 layers on Hierarchical Equality
- A GruCELL based RNN on Binary Addition
- Gemma-2-2B on MCQA
Across all experimental configurations, PLOT is usually worse than DAS, but only barely and in a few configurations (such as the majority of AT rows for MCQA) it is consistently better. It is always much faster, and for the latter 2 experiments, the authors also propose a variant, PLOT-DAS, which first uses PLOT for coarse localization and then trains DAS only within selected states. The biggest performance gap is in the MCQA setting, where PLOT-DAS ranks 0.02 below full DAS but runs 20 times as fast.
The authors conclude the paper by stating that sharper localization rules may improve joint correspondence. They also note that unlike DAS, which must operate in the natural neural bases' of the network being analyzed, PLOT can work on interpretable feature constructions such as cross-layer transcoders or, for instance, models derived from parameter decomposition.
Critical strengths
Formalization of causal localization as a problem of optimal transport: Highly novel and interesting. PLOT and PLOT-DAS for joint correspondence: The described algorithm performs in general as well as DAS on the experiments, is much faster, and has more flexibility in being applicable also to things like transcoders. This gives much more utility than DAS itself and opens the door to usage in much larger models.
Critical weaknesses
Justification for optimal transport: While the authors do make good arguments for OT, there is still some unresolved tension. Simpler independent matching methods are quite competitive with OT, and it is not clear to me that optimal transport is the main source of the increase in performance. It would be nice to compare PLOT-DAS against DAS guided by one of the simpler localization strategies. Experimental breadth: This is a bit of a nitpick, but it would be nice to see a few more experiments on a task slightly more complex than MCQA. Note that I do not actually consider this necessary for acceptance, I just think it would slightly improve the paper.
Questions for authors
Could you elaborate on our justification for optimal transport? In particular, I'm interested in:
- Given the performance of the non-OT methods, how much of the performance can we really attribute to OT?
- Why do the authors expect that the causal correspondence problem has the constraints imposed by optimal transport? For instance, an abstract variable might be distributed over multiple sites.
4 comments
I thank the authors for their response! Especially the last point; it seems like I partly misunderstood part of the algorithm. Thank you for the clarification.
Overall, depending on the result of the extra experiments and the insights, I could justify going to a Clear Accept. For now I retain my score, but I also want to make it clear to the ACs and PCs (and authors), that I do believe this paper warrants publication in its current form.
We are glad that our response helped clarify this point, and very much appreciate your positive outlook and support for our paper. Since posting our response, have begun work on the immediate extensions of the experiments already in the paper: DAS guided by non-OT methods, Boundless DAS comparisons, and fuller runtime accounting. After that, we plan to work on an additional task, such as ARC-Challenge or RAVEL. We hope to share follow-up results soon during the discussion period and, if progress and time permit, results from an additional task as well.
Thank you again for your constructive feedback and support.
Follow-up: New screening and Boundless DAS results
Following up on our previous update, we can now share the completed non-OT-guided DAS and full Boundless DAS comparisons on MCQA, together with their runtime results.
The first five methods in the table below were run on NVIDIA H100 80GB GPUs using Gemma-2-2B, five seeds, and identical MCQA train/calibration/test pair banks. Measured values are mean ± sample SD, with IIA averaging answer pointer (AP) and answer token (AT) within each seed. Layers are zero-indexed.
| Method | Test IIA (%) | Serial runtime (min/seed) | Seed 0 AP: layer / dim | Seed 0 AT: layer / dim |
|---|---|---|---|---|
| PLOT-DAS | 87.45 ± 3.19 | 9.83 ± 0.81 | L17 / 1152 | L24 / 128 |
| Cosine matching + DAS | 86.45 ± 3.41 | 9.45 ± 0.73 | L19 / 256 | L24 / 128 |
| Lowest-cost + DAS | 86.45 ± 3.41 | 9.66 ± 1.24 | L19 / 256 | L24 / 128 |
| IIA matching + DAS | 87.40 ± 3.81 | 10.09 ± 1.24 | L18 / 512 | L24 / 128 |
| Full Boundless DAS | 88.35 ± 1.88 | 71.18 ± 3.83 | L17 / 334 | L22 / 420 |
| Full DAS (paper; time estimated) | 89.60 ± 2.13 | 198 (est.) | L17 / 32 | L20 / 512 |
Matching formulation and alternatives.
As discussed in Appendix B.1 (Non-OT Matching Alternatives), we adopt OT because it provides a principled global constrained matching formulation, accommodates marginal relaxation through UOT, and draws on well-developed statistical and computational foundations. OT provides a coherent correspondence formulation, but non-OT alternatives can also be competitive in practice. We extend the appendix’s cosine and IIA matching to layer screening before DAS, and add lowest-cost screening using squared distances without transport, as proposed by Reviewer AvSU. Given the complementary roles of screening and subspace learning, the choice of matching is a flexible component of our framework.
The table shows that the four screening methods achieve similar mean test IIA, within 1.00 percentage point of each other, with PLOT-DAS having the highest mean at 87.45%. Their serial runtimes are also close, ranging from 9.45 to 10.09 min/seed, a difference of about 38 seconds. These results support the flexibility of the matching framework.
Boundless DAS (bDAS).
Runtime follows Appendix A.2; the MCQA procedures are in Appendix A.5 (Algorithms 11–12). Full bDAS adapts Algorithm 12 to learn the subspace dimension instead of searching over fixed sizes like standard/full DAS. In these experiments, standard DAS stops training when improvement plateaus. bDAS instead trains for 50 epochs while gradually sharpening a soft selection of directions. At evaluation, each direction is either included in the intervention or excluded, yielding the eventual subspace.
The first five rows are measured on H100s, while the full-DAS accuracy and handles come from the paper’s A40 runs. Its H100 time is estimated by min, scaling by the observed PLOT-DAS hardware speedup. We provide this estimate for a timely comparison, but emphasize that it is not a measurement of a full H100 sweep. We plan to rerun full DAS on H100s for the final version and add method-by-method operation counts and a computational-complexity discussion.
Relative to the estimated full-DAS runtime, full bDAS (without screening) is approximately faster, taking 71.18 min per seed. In comparison, the guided-DAS methods take 9.45–10.09 min per seed, making them – faster than full bDAS. For PLOT-DAS specifically, this saves about an hour per seed, with mean IIA less than one percentage point lower: 87.45% versus 88.35%. Across all guided-DAS variants, the gap is 0.90–1.90 percentage points. Screening therefore preserves strong intervention accuracy while sharply reducing serial runtime, even when considering full bDAS.
Combining the strengths of screening and bDAS.
As proposed in our initial response to Reviewer rbT5, a natural next step is to use matching to select the layer, then let bDAS learn the dimension. This would avoid both the full layer sweep in bDAS and the ten-dimension search in PLOT-DAS. We plan to evaluate PLOT-guided and non-OT-guided bDAS in the final version, and expect the combination to further improve the accuracy–runtime tradeoff.
We are currently working on extending evaluation to additional tasks such as ARC-Challenge or RAVEL, as outlined in our previous update, and preparing a consolidated comparison on common hardware for the final version, although these results will likely be available only after the discussion period concludes. We will also revise the presentation as outlined in our initial response, clarifying the matching formulation and the complementary roles of screening and subspace learning. We sincerely thank you for your time and valuable feedback.
We hope these additional experimental results and the insights they provide address your remaining questions and support the stronger assessment you mentioned in your previous response.
Thank you very much for the extensive comments and feedback. We appreciate the time you took to make explicit your concerns about our paper. We address each of your concerns individually below.
Justification for optimal transport: While the authors do make good arguments for OT, there is still some unresolved tension. Simpler independent matching methods are quite competitive with OT, and it is not clear to me that optimal transport is the main source of the increase in performance. It would be nice to compare PLOT-DAS against DAS guided by one of the simpler localization strategies.
Thank you for the positive outlook on our paper and the constructive feedback. We agree that the current comparisons do not isolate how much of PLOT-DAS’s benefit comes from OT itself, rather than from a matching-based screening of locations before subspace learning. We focus the paper around OT because it provides a principled joint matching formulation and performs well empirically. However, the matching rule is a flexible component of the approach, as noted in Remark 3.1, and we will gladly add experiments to the appendix comparing PLOT-DAS with DAS guided by non-OT localization, such as those considered in Appendix B.1. In particular, we will keep the candidate sites, data splits, and downstream DAS procedure fixed so that the comparison isolates the choice of localization strategy. If we obtain these results during the discussion period, we will update the reviewer here with the findings and resulting insights.
Experimental breadth: This is a bit of a nitpick, but it would be nice to see a few more experiments on a task slightly more complex than MCQA. Note that I do not actually consider this necessary for acceptance, I just think it would slightly improve the paper.
Thank you for the suggestion. We agree that additional experiments will further strengthen the paper. After addressing the comments concerning our current experiments, we will work to add an experiment on ARC-Challenge from MIB, which involves answering grade-school multiple-choice science questions requiring knowledge beyond the information provided in the prompt (MIB, Section 2.1). Another option we will consider is the RAVEL task from MIB, which tests whether neural interventions can selectively change a model’s answers about one attribute of a city, such as its country, while preserving its answers about other attributes. These results will likely be included in the appendix.
Given the performance of the non-OT methods, how much of the performance can we really attribute to OT?
Thanks again for this excellent question. Please see our response to the first Critical Weakness above. We believe that the additional comparison with DAS guided by non-OT localization will help distinguish the benefit of screening itself from any additional benefit provided specifically by OT, and plan to discuss the resulting insights in the revision.
Why do the authors expect that the causal correspondence problem has the constraints imposed by optimal transport? For instance, an abstract variable might be distributed over multiple sites.
We hope that we understand the reviewer’s point correctly, but please let us know if this is not the case. In general, an abstract variable being distributed across multiple neural sites is fully compatible with PLOT. Each row of the coupling can assign mass to several sites, and the resulting handle combines interventions at those sites, potentially across layers (based on calibration). Different abstract variables can also assign mass to the same site. OT does not require a one-to-one correspondence. For example, in 1-stage PLOT PCA for binary addition, both C2 and C3 carry variables were matched to the same PCA component in the same timestep.
The use of uniform reference weights is a modeling choice. Uniform weights avoid an a priori preference for particular variables or sites. However, enforcing a uniform neural marginal also constrains how total matching mass is distributed across sites. This couples the localization decisions across variables, but can be restrictive when sites differ in relevance or are shared unevenly among variables. This motivates our use of one-sided UOT in MCQA, which retains the abstract marginal while allowing nonuniform total mass on neural sites.