acceptodds
Under review as a conference paper at ICLR 2027

The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoning

Abstract

Transformer-based language models perform well on symbolic tasks, yet it remains unclear whether they learn generalizable rules or rely on statistical shortcuts. Mechanistic studies link algorithmic behavior to structured internal representations, motivating the hypothesis that robust reasoning benefits from separating values from the types that control their manipulation. Can making this separation an architectural primitive improve the learnability and generalization of logical mechanisms? We introduce **STRAT** (**ST**ratified **R**egisters **A**nd **T**ypes), which partitions the residual stream into orthogonal Data and Type subspaces and uses Type-based attention and gating to govern Data transformations. Controlled arithmetic ablations identify three failure modes associated with data-control interference: the Linear Trap, Gradient Wall, and Open Gate Trap. Mechanistic analysis reveals interpretable logical structure, and in arithmetic, STRAT reduces median out-of-distribution (OOD) error 35-fold relative to a Transformer baseline. On each of 11 datasets spanning 10 tasks, STRAT outperforms the Transformer baseline in mean accuracy, by 26 percentage points on average, with both models trained from 10 base examples per dataset using identical task-specific augmentation where applicable. Under distribution shift, STRAT's mean accuracy drops by only 2.39 percentage points, compared with 11.75 for the Transformer.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Model Architecture, Representation Learning, Logical Reasoning, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_CeqxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new architecture, STRAT, which is based on the idea of separating data and types in their embedding subspaces orthogonally as an architecturally imposed inductive bias. The authors motivate the work by investigating failure modes with standard transformers and identify three failure modes: the Linear Trap, Gradient Wall, and Open Gate Trap. The authors show benefits of STRAT compared to the transformer on different algorithmic reasoning tasks, with particularly strong improvements for out-of-distribution evaluation .

Critical strengths
  • Thoughtful motivation and experimental design: The authors introduce the paper and the motivation for the work from a very clear angle: how architectural inductive biases can influence the capability of a model to learn logical reasoning. The author's design for STRAT is carefully thought out: first implementing an arithmetic algorithm with a standard transformer in Section 3 to show this is possible and then investigate why this algorithm could be difficult to learn in gradient descent in Sectino 4. This motivates the whole work and the need for the STRAT architecture.

  • Empirical Results: Table 6 shows clear improvements over the standard architecture on a wide suite of algorithmic tasks, showing that STRAT performs better in-distribution and out-of-distribution. The out-of-distribution performance is particularly strong, as it shows that the architectural inductive bias is providing generalisation beyond what the evaluated standard transformer achieves through gradient descent.

Critical weaknesses
  • Predefined Types: Section 7's experiments use predefined task-specific typing codings. Even though Section 5.4 motivates and demonstrates that strat can learn meaningful representations for types. Evaluating learned types in Section 7's experiments would strengthen the claims about the generalisation capabilities of STRAT. Also, it is unclear if the geometric structures observed in the paper are consistent across seeds. Do they always appear or are they specific to the seeds that the authors used?
  • Support for claims of limitations of standard transformers: The paper is motivated by inherented limitations of standard transformers, but the empirical evidence is only based on six small architectures and a small training budget. Are these failures truly general to the transformer family (i.e., would it persist with different transformer architectures or different-sized models, or alternative training settings like changing the optimizer)?
Questions for authors
  • Is it possible for the authors to evaluate learned type representations against the predefined ones on the tasks in Section 7?
  • In regard to the limitation of standard transformers, could the authors discuss if the failures in Section 4 persist across different settings? What evidence then supports this claim that it is inherent to the standard transformer architecture and not just the six models evaluated?
6 comments
AuthorsOpenReview ↗

Response to Reviewer Ceqx (1/3)

Dear Reviewer Ceqx:

Thank you very much for appreciating the thoughtful motivation and experimental design of our work. We are particularly encouraged by your recognition of STRAT's clear improvements over the standard architecture, especially its strong out-of-distribution performance. Below, we address your specific comments and questions point by point (the corresponding revisions have been included in the uploaded revised version).

AuthorsOpenReview ↗

Response to Reviewer Ceqx (2/3)

W1 - Predefined Types: Section 7's experiments use predefined task-specific typing codings. Even though Section 5.4 motivates and demonstrates that strat can learn meaningful representations for types. Evaluating learned types in Section 7's experiments would strengthen the claims about the generalisation capabilities of STRAT.

Also, it is unclear if the geometric structures observed in the paper are consistent across seeds. Do they always appear or are they specific to the seeds that the authors used?

Q1 - Is it possible for the authors to evaluate learned type representations against the predefined ones on the tasks in Section 7?

Thank you for these helpful suggestions and for giving us an opportunity to further strengthen the claims about the generalisation capabilities of STRAT. Following your suggestions, we evaluated learned Type representations against the predefined ones on all 11 datasets of Section 7, and examined whether the learned geometric structures are consistent across seeds.

For experiments config: all data splits, ID and OOD test sets, model architectures, training settings, hyperparameters, and seeds 41–50 are the same as in Section 7. The predefined Type is replaced by a learnable embedding layer with one vector per token kind. The embedding layer is initialized from N(0, 0.02), the same as in the arithmetic experiments. We set the embedding dimension to 5 for all datasets for consistency, except for Sudoku, where we keep the dimension of its original predefined Type (37) because Sudoku is the most complex task and has the largest vocabulary (each cell as a token).

Table a: Accuracy (%) of STRAT with the predefined Type and with the learned Type on the 11 datasets of Section 7 (mean ± SEM over 10 seeds).

DatasetPredefined Type (ID)Predefined Type (OOD)Learned Type (ID)Learned Type (OOD)
SATNet Sudoku86.58 ± 1.0883.80 ± 1.3089.87 ± 1.0886.70 ± 1.31
SATBench92.33 ± 1.3690.50 ± 2.1592.11 ± 1.4590.66 ± 1.57
SATLIB99.98 ± 0.0299.96 ± 0.04100.00 ± 0.00100.00 ± 0.00
Tic-Tac-Toe Endgame98.19 ± 0.0094.64 ± 0.0097.96 ± 0.2394.46 ± 0.18
Game of Life88.71 ± 2.7279.26 ± 12.9987.57 ± 3.0877.16 ± 12.67
PMLB Parity596.94 ± 0.0697.12 ± 0.2597.25 ± 0.3197.44 ± 0.38
PMLB M-of-N100.00 ± 0.00100.00 ± 0.00100.00 ± 0.00100.00 ± 0.00
PMLB MONK-290.86 ± 3.0490.92 ± 3.0798.28 ± 1.1696.13 ± 2.29
PMLB LED24100.00 ± 0.00100.00 ± 0.0095.10 ± 4.9095.36 ± 4.64
PMLB ThreeOf980.18 ± 0.5773.12 ± 0.88100.00 ± 0.00100.00 ± 0.00
SPECT Heart94.91 ± 1.0593.05 ± 0.15100.00 ± 0.00100.00 ± 0.00
Average93.5191.1396.1994.36

As shown in Table a, STRAT with the learned Type performs on par with STRAT with the predefined Type: the average accuracy is 96.19 / 94.36 (ID / OOD), compared with 93.51 / 91.13, and no dataset decreases by more than two SEM. This shows that the Type embeddings do not need to be predefined: they can be learned directly from the task.

To further examine whether the learned geometric structures are consistent across seeds, we analyzed the learned Type embeddings on Sudoku with 10 seeds, where the type of a cell is its row, column, and box, and the model is never told any of them. Figure 4 in the revised paper (Appendix G.1) shows the cosine similarity between the learned Type vector of each cell and that of a reference cell, averaged over the 10 seeds. Every board shows a red cross centered on the reference cell: the cells in its row, column, and box have similar Type vectors (red), while all other cells do not (blue). Specifically, the mean cosine similarity to the reference cell is +0.30 for cells in its row, +0.31 in its column, and +0.19 in its box, compared with −0.11 for all other cells (all about 0 before training). Importantly, the same pattern appears in every one of the 10 seeds, as shown in Figure 5 in the revised paper (Appendix G.1). Thus, this structure is not specific to particular seeds: in every seed, the learned Type learns by itself the row, column, and box structure encoded in the predefined Type.

Thank you again for these helpful suggestions. We have included these results and the analysis in the revised version (Section 7 and Appendix G).

AuthorsOpenReview ↗

Response to Reviewer Ceqx (3/3)

W2 - Support for claims of limitations of standard transformers: The paper is motivated by inherented limitations of standard transformers, but the empirical evidence is only based on six small architectures and a small training budget. Are these failures truly general to the transformer family (i.e., would it persist with different transformer architectures or different-sized models, or alternative training settings like changing the optimizer)?

Q2 - In regard to the limitation of standard transformers, could the authors discuss if the failures in Section 4 persist across different settings? What evidence then supports this claim that it is inherent to the standard transformer architecture and not just the six models evaluated?

Thank you for raising this important concern and giving us an opportunity to further clarify the scale of our experiments and to strengthen the evidence with additional experiments.

In Section 4, we trained 12 configurations of the Transformer block, each with 50 seeds (600 runs in total; Section 4.1). These configurations change the input scaling, normalization, curriculum, regularization, and initialization, and none of the 600 runs succeeds. In addition, the Transformer baselines in Section 7 have 1 to 16 layers and are trained with Adam or AdamW (Appendix D). Across the 11 datasets, their accuracy drops by 11.75 points under distribution shift, compared with 2.39 for STRAT (Table 6).

To further support that the failure comes from the architecture, we conducted two additional experiments. First, we measured how each trained model fails. In 10 of the 12 configurations the failure has the same form: the models use the operators correctly at training magnitudes but ignore them at OOD magnitudes. Second, we conducted a full factorial ablation: with the same data, optimizer, and training budget, we change only the connections of the standard block. When the gate and attention read only the Type, 167 of 200 runs succeed (84%), compared with 40 of 1,400 (2.9%) for all other combinations. Thus, the failure follows the connections that let operand values reach the control computation, rather than a particular model or seed. We have included these experiments in the revised paper (see Section 4.3, Table 2, Section 5.4, and Appendix B.5).

Thank you again for this valuable comment.

Reviewer_roH5OpenReview ↗
Rating
4(+2)
Confidence
4
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates why small transformer models can struggle to learn generalizable rules in controlled logical and discrete reasoning tasks (when trained from scratch on these tasks). Through controlled experiments on arithmetics, the authors identify potential failure modes in which transformers rely on shortcuts or simply struggle to learn the appropriate attention and gating mechanisms, and analyze the circuits learned by successful models.

Motivated by these findings, the authors propose STRAT, a family of transformer variants that explicitly separate type representations (how a computation should proceed, e.g. which arithmetic operation should be performed) from data values (intermediate representations being manipulated). They use the type representations to guide attention and condition how values get processed. This architectural modification aims to reduce interference between numerical information and type signals, providing a favorable inductive bias for discrete reasoning. Experiments on several datasets with reasoning-heavy tasks demonstrate substantial improvements over basic transformer baselines, particularly under low-data and distribution shift regimes.

Critical strengths

This paper has two main positive contributions. First, it analyzes why small transformers trained from scratch on discrete reasoning tasks struggle to learn the logic rules in clean controlled settings, going beyond the typical "observing poor OOD performance" in LLM reasoning papers. Second, it develops architectural modifications motivated by those analysis insights. Let me elaborate below.

This paper conducts a mechanistic case study of why a small transformer has the expressive power to represent an arithmetic algorithm (like addition and subtraction), yet it struggles to learn the algorithm in real training. The main hypothesis is that, mixing the numerical values in the problem with the signals specifying what operation (the "type" signal) to apply promotes harmful interference and shortcuts, resulting in poorer OOD performance, which is a very reasonable hypothesis. The authors do several controlled experiments to support the hypothesis (section 4.2, appendix B and Table 2), by holding different parts of the transformer fixed to the “ideal” pattern and testing whether the rest of the components can learn the appropriate mechanisms.

Motivated by the interference/entanglement and shortcut hypothesis, the authors build the STRAT family of transformer variants that restrict attention and gating mechanisms to better disentangle information that controls computation (type) from the computational state being manipulated (value). The gains are reported to be significant across the datasets, with results on Sudoku being especially impressive (from ~10% to 84%). They also do some case studies showing that that the modified model approximately implements the disentangled operator vs operand circuits in the arithmetic task, which is quite interesting.

In sum, at the technical level, I can see the paper having a coherent flow of mechanistic analysis of failure modes →\to architectural modifications, which I think is valuable and less common in the mechanistic intepretability community.

Critical weaknesses

Overclaims. The three specific failure mechanisms of the base transformer which the authors propose ("the Linear Trap, the Gradient Wall, and the Open Gate Trap"), are more of the "suggestive interpretations" nature than disentangled mechanisms with causal evidence. The experiments include meaningful interventions, but the diagnoses often rely on prediction errors rather than direct measurements of the proposed mechanisms.

For instance, in the arithemtic setting, poor extrapolation after initializing gate biases to positive values is interpreted as evidence that the gates remained open. This is a problematic leap: prediction error alone does not establish this internal mechanism. Setting a positive gate bias only specifies an initial condition, but I cannot find in the paper whether the trained gates actually remain open (or at least showing whether their outputs differ appropriately between addition and subtraction).

Writing. The paper is quite dense, and some key points are only presented in the appendix. For instance, Section 3's actual content is almost all in Appendix A, it is very hard to understand the theoretical point being made about the model's expressive power by Section 3 alone; another example is Section 4.2, which is supposedly about analyzing failure taxonomies of the base transformer, but leaves nearly all interpretations to the appendix, only with a Table 2 dangling in the main text that is very hard to understand by itself. While I understand that the paper has a lot of results to present, I honestly struggled a lot to follow the paper's claims and their evidence on a first read, having to piece scattered pieces across the main text and appendix, so I consider this a significant weakness of the paper. There likely is a need to re-prioritize what to present in the main text, so the reader can understand what the main narrative and takeaways are, and readers interested in a specific technical detail or extension can refer to the appendix.

Questions for authors

Please refer to my weakness section. I think this work is technically quite promising, and would be happy to raise my score if the authors can better flesh out and be more rigorous in their mechanistic analysis parts, and address the writing clarity problems.

7 comments
AuthorsOpenReview ↗

Response to Reviewer roH5 (1/3)

Dear Reviewer roH5:

Thank you very much for appreciating our mechanistic analysis of why small Transformers struggle to learn logic rules, and the coherent flow from this analysis to our architectural modifications. We are particularly encouraged by your view that this work is technically quite promising. Below, we address your concerns regarding the rigor of our mechanistic analysis and the writing clarity (all revisions mentioned below have been incorporated into the uploaded revised paper).

AuthorsOpenReview ↗

Response to Reviewer roH5 (2/3)

W1 - Overclaims. The three specific failure mechanisms of the base transformer which the authors propose ("the Linear Trap, the Gradient Wall, and the Open Gate Trap"), are more of the "suggestive interpretations" nature than disentangled mechanisms with causal evidence. The experiments include meaningful interventions, but the diagnoses often rely on prediction errors rather than direct measurements of the proposed mechanisms. For instance, in the arithemtic setting, poor extrapolation after initializing gate biases to positive values is interpreted as evidence that the gates remained open. This is a problematic leap: prediction error alone does not establish this internal mechanism. Setting a positive gate bias only specifies an initial condition, but I cannot find in the paper whether the trained gates actually remain open (or at least showing whether their outputs differ appropriately between addition and subtraction).

Thank you for this insightful and detailed comment. Direct measurements of the proposed mechanisms are valuable for improving the rigor of our mechanistic analysis and for providing direct evidence for each failure mode.

Following your comment, as well as a similar one from another reviewer, we have devised a set of measurements and thresholds to test all those assumptions. These measurements and their results are now reported in Section 4, and Appendix B gives all the details. You are correct that most of the initial intuitive interpretations do not hold. Nevertheless, we decided to keep them as hypotheses, some of which hold and most of which fail. The measurements reveal a much more interesting landscape of failure modes, some of which we still don't have explanations for. But most of the failures have two sources: a gate that reads the operand magnitude, directly or through attention, and operator effects learned as fixed offsets. In both, the computation that decides how to treat an operand depends on the operand's value, which is the interference that STRAT is designed to remove.

Thank you again for this valuable comment. We have included these measurements in the revised version (Section 4 and Appendix B) to support the analysis of each failure with direct evidence.

AuthorsOpenReview ↗

Response to Reviewer roH5 (3/3)

W2 - Writing. The paper is quite dense, and some key points are only presented in the appendix. For instance, Section 3's actual content is almost all in Appendix A, it is very hard to understand the theoretical point being made about the model's expressive power by Section 3 alone; another example is Section 4.2, which is supposedly about analyzing failure taxonomies of the base transformer, but leaves nearly all interpretations to the appendix, only with a Table 2 dangling in the main text that is very hard to understand by itself. While I understand that the paper has a lot of results to present, I honestly struggled a lot to follow the paper's claims and their evidence on a first read, having to piece scattered pieces across the main text and appendix, so I consider this a significant weakness of the paper. There likely is a need to re-prioritize what to present in the main text, so the reader can understand what the main narrative and takeaways are, and readers interested in a specific technical detail or extension can refer to the appendix.

Thank you for this helpful suggestion and for your careful reading of our paper. Re-prioritizing what is presented in the main text is valuable for improving the readability of the paper. Following this suggestion, we revised Sections 3 and 4 so that their main points can be followed from the main text.

Regarding Section 3, its purpose is to show that a single Transformer block is able to express signed arithmetic, so the failures in Section 4 do not come from a lack of expressive power. To make this point easier to follow, we added a worked example (see Table 1 in the revised paper) that follows one expression through the hand-coded block step by step. This allows readers to understand how the construction works from the main text, while the full weight matrices remain in Appendix A.

Regarding Section 4, the main text now explains the measurements reported in Table 2 and states the main findings directly. Thus, Table 2 can be understood together with the text, while the technical details are provided in Appendix B for interested readers.

Thank you again for this very helpful suggestion. We have included these changes in the revised version to make the main narrative and takeaways easier to follow.

Reviewer_yNFuOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

With hand-crafted weights, a transformer can achieve perfect accuracy on an arithmetic task. However, learning with gradient descent using 128K samples does not generalize to OOD problems. The authors insist that this originates from interference between value and type, causing prediction shortcuts or optimization failures. The proposed new architecture, called “STRAT”, has an inductive bias that addresses this problem; it separates activation vectors into value space and type space, and constrains type-to-value intervention to occur only through GLU. The authors insist that natural relationships between types are obtained when embeddings for type encoding are learned. Also, on the arithmetic task with 10 training samples, STRAT learns type embeddings spontaneously and demonstrates much better OOD performance than basic transformers. Furthermore, with hand-crafted type embeddings, the authors evaluate STRAT on 11 datasets. STRAT preserves OOD accuracy with only about 2% degradation, compared to 12% for a basic transformer.

Critical strengths
  1. The experimental results are very strong; the improvements in OOD performance achieved by the new architecture are huge.
  2. The writing is easy to read. Each sentence is not too long, and the wording is easy to follow.
Critical weaknesses
  1. Limited experiments on versatility: In the experiments, value embeddings are predefined rather than learned. Also, there is no evaluation of STRAT with learned value embeddings for diverse tasks. The lack of these results suggests the need to provide hand-crafted embeddings for tasks and limits the contribution of this paper to architecture studies.
  2. Ambiguity in the explanation of failure modes: Section 4.2 provides an overview of each failure mode (“Linear Trap”, “Gradient Wall”, etc.), and Appendix B.2 provides more details about them. However, each interpretation remains an assumption and has not been validated.
Questions for authors
  1. Are there any experiments on learned embeddings and/or failure analyses related to what I mentioned in the first weakness?
  2. The “plus” embedding and the “minus” embedding reported in Table 3 have slightly different norms. Have any specific effects due to this norm difference been confirmed?
7 comments
AuthorsOpenReview ↗

Response to Reviewer yNFu (1/4)

Dear Reviewer yNFu:

Thank you very much for appreciating the strength of our experimental results and the readability of our writing. We are particularly encouraged by your recognition of the huge improvements in OOD performance achieved by the new architecture. Below, we address your specific comments and questions point by point (all revisions mentioned below have been incorporated into the uploaded revised paper).

AuthorsOpenReview ↗

Response to Reviewer yNFu (2/4)

W1 - Limited experiments on versatility: In the experiments, value embeddings are predefined rather than learned. Also, there is no evaluation of STRAT with learned value embeddings for diverse tasks. The lack of these results suggests the need to provide hand-crafted embeddings for tasks and limits the contribution of this paper to architecture studies.

Q1 - Are there any experiments on learned embeddings and/or failure analyses related to what I mentioned in the first weakness?

Thank you for raising this and giving us an opportunity to further demonstrate the versatility of STRAT and to strengthen its contribution to architecture studies. Following your comments, we re-evaluated all 11 datasets of Section 7 with learned Type embeddings.

For experiments config: all data splits, ID and OOD test sets, model architectures, training settings, hyperparameters, and seeds 41–50 are the same as in Section 7. The hand-crafted Type is replaced by a learnable embedding layer with one vector per token kind. The embedding layer is initialized from N(0, 0.02), the same as in the arithmetic experiments. We set the embedding dimension to 5 for all datasets for consistency, except for Sudoku, where we keep the dimension of its original hand-crafted Type (37) because Sudoku is the most complex task and has the largest vocabulary (each cell as a token).

Table A: Accuracy (%) of STRAT with the hand-crafted Type and with the learned Type on the 11 datasets of Section 7 (mean ± SEM over 10 seeds).

DatasetHand-crafted Type (ID)Hand-crafted Type (OOD)Learned Type (ID)Learned Type (OOD)
SATNet Sudoku86.58 ± 1.0883.80 ± 1.3089.87 ± 1.0886.70 ± 1.31
SATBench92.33 ± 1.3690.50 ± 2.1592.11 ± 1.4590.66 ± 1.57
SATLIB99.98 ± 0.0299.96 ± 0.04100.00 ± 0.00100.00 ± 0.00
Tic-Tac-Toe Endgame98.19 ± 0.0094.64 ± 0.0097.96 ± 0.2394.46 ± 0.18
Game of Life88.71 ± 2.7279.26 ± 12.9987.57 ± 3.0877.16 ± 12.67
PMLB Parity596.94 ± 0.0697.12 ± 0.2597.25 ± 0.3197.44 ± 0.38
PMLB M-of-N100.00 ± 0.00100.00 ± 0.00100.00 ± 0.00100.00 ± 0.00
PMLB MONK-290.86 ± 3.0490.92 ± 3.0798.28 ± 1.1696.13 ± 2.29
PMLB LED24100.00 ± 0.00100.00 ± 0.0095.10 ± 4.9095.36 ± 4.64
PMLB ThreeOf980.18 ± 0.5773.12 ± 0.88100.00 ± 0.00100.00 ± 0.00
SPECT Heart94.91 ± 1.0593.05 ± 0.15100.00 ± 0.00100.00 ± 0.00
Average93.5191.1396.1994.36

As shown in Table A, STRAT with the learned Type performs on par with STRAT with the hand-crafted Type: the average accuracy is 96.19 / 94.36 (ID / OOD), compared with 93.51 / 91.13, and no dataset decreases by more than two SEM. This shows that the Type embeddings do not need to be hand-crafted: they can be learned directly from the task.

To further examine whether the learned Type really learns the underlying type of each token, as in Section 5.5(in revised submission), we analyzed the learned Type embeddings on Sudoku, where the type of a cell is its row, column, and box, and the model is never told any of them. Figure 4 in the revised submission (Appendix G.1) shows the cosine similarity between the learned Type vector of each cell and that of a reference cell. To ensure consistency, we ran 10 seeds and report the average. Every board shows a red cross centered on the reference cell: the cells in its row, column, and box have similar Type vectors (red), while all other cells do not (blue). Specifically, the mean cosine similarity to the reference cell is +0.30 for cells in its row, +0.31 in its column, and +0.19 in its box, compared with −0.11 for all other cells (all about 0 before training). The same pattern appears in every seed, as shown in Figure 5 in the revised paper (Appendix G.1). Thus, the learned Type learns by itself the row, column, and box structure that we previously encoded by hand.

Thank you again for this helpful comment. We have included these results and the analysis in the revised version (Section 7 and Appendix G) to better demonstrate the versatility of STRAT.

AuthorsOpenReview ↗

Response to Reviewer yNFu (3/4)

w2 - Ambiguity in the explanation of failure modes: Section 4.2 provides an overview of each failure mode (“Linear Trap”, “Gradient Wall”, etc.), and Appendix B.2 provides more details about them. However, each interpretation remains an assumption and has not been validated.

Thank you for this thoughtful comment on validating each failure mode with direct evidence, which is valuable for improving the rigor of our mechanistic analysis and strengthening the motivation for the STRAT architecture.

Following your comment, as well as a similar one from another reviewer, we have devised a set of measurements and thresholds to test all those assumptions. These measurements and their results are now reported in Section 4, and Appendix B gives all the details. You are correct that most of the initial intuitive interpretations do not hold. Nevertheless, we decided to keep them as hypotheses, some of which hold and most of which fail. The measurements reveal a much more interesting landscape of failure modes, some of which we still don't have explanations for. But most of the failures have two sources: a gate that reads the operand magnitude, directly or through attention, and operator effects learned as fixed offsets. In both, the computation that decides how to treat an operand depends on the operand's value, which is the interference that STRAT is designed to remove.

Thank you again for this valuable comment. We have included these measurements in the revised version (Section 4 and Appendix B) to support the analysis of each failure with direct evidence.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to PCs, SACs, ACs, and Reviewers

Dear PCs, SACs, ACs, and Reviewers,

We sincerely thank you for the time and effort you have spent on our work. Your comments have helped us strengthen our mechanistic analysis and further demonstrate the versatility of STRAT. To facilitate the assessment of our work, we summarize below the strengths recognized by the reviewers and the main points of our responses and revisions.

We are grateful and encouraged that the reviewers highlighted the following strengths:

  • Very strong experimental results, with large improvements in OOD performance over the standard architecture (Reviewers yNFu and Ceqx).
  • A clear and easy-to-read presentation (Reviewer yNFu).
  • A technically promising approach to learning logic rules with Transformers (Reviewer roH5).
  • A coherent flow from the mechanistic analysis of failure modes to the architectural modifications, which is valuable and less common in the mechanistic interpretability community (Reviewer roH5).
  • A thoughtful motivation and experimental design (Reviewer Ceqx).

Based on the reviewers' suggestions, our responses and revisions focus on the following points:

  • Learned Type evaluation (Reviewers yNFu and Ceqx). We re-evaluate all 11 datasets of Section 7 with the predefined Type replaced by a learnable embedding layer with one vector per token kind, and analyze the learned Type embeddings on Sudoku across 10 seeds, providing evidence that the Type can be learned from the task and that the learned structure is consistent across seeds.

  • Norm difference between Plus and Minus (Reviewer yNFu). We measure the effect of this difference with additional experiments and add an explanation of its cause to Section 5.5.1.

  • Direct evidence for the failure modes (Reviewers yNFu and roH5). We devise a set of measurements and thresholds to test each proposed failure mode and rewrite Section 4 around the results, which revise our initial interpretations and trace most failures to the interference that STRAT is designed to remove.

  • Main text and appendix (Reviewer roH5). We revise Sections 3 and 4 so that their main points can be followed from the main text:

  • Generality of the Transformer failures (Reviewer Ceqx). We clarify the scale of our experiments and present a full factorial ablation of the standard block, showing that the failures follow the connections that let operand values reach the control computation, rather than a particular model or seed.

All of these changes have been included in the uploaded revised paper (Sections 4, 5.4, 5.5.1, and 7; Appendices B and G).

We hope that this overview, along with our point-by-point responses, helps the ACs and reviewers assess the contribution of STRAT and the evidence that supports it. We sincerely thank the reviewers, ACs, SACs, and PCs for their time and thoughtful feedback, which have greatly improved our manuscript.

Best regards,

The Authors

Dear Reviewers, as the rebuttal deadline is approaching, please actively engage in the discussion based on the authors' responses. Thank You!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.