acceptodds
Under review as a conference paper at ICLR 2027

Tracing the Causal Transition from Event Relations to Moral Judgment in Language Models

14 views

Abstract

Large language models (LLMs) exhibit competence in moral reasoning on behavioral benchmarks, yet whether this reflects actual moral computation or statistical shortcuts remains unclear. To bridge this gap, we propose a bottom-up interpretive paradigm that moves beyond static value-principle matching. Grounded in the Theory of Dyadic Morality (TDM), this paradigm traces the causal-mechanistic trajectory through which LLMs internally arrive at moral judgments. Models first spontaneously form event relations and then use them to determine how outcome information conditions judgments about the moral agent, a process we term Event-Navigated Causal Transition (ENaCT). We substantiate this process through an initial causal abstraction followed by step-by-step causal interventions. Specifically, we abstract moral events into structural causal graphs spanning distinct event relations and use cross-domain instantiation to build MoralTrace, which provides isomorphic counterfactual pairs for fine-grained manipulations. Then, we use staged causal interventions to disentangle the three operators underlying the progression from moral-patient completion, through intention inference, to outcome-conditioned judgment. At the neural level, circuit discovery and feature decomposition show that ENaCT's behavior arises from distributed representations converging onto a low-dimensional latent space, revealing strong generalization across diverse task settings. Together, these results uncover a causally grounded mechanism of moral reasoning in LLMs and reveal emergent patterns across model scales and training stages.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, large language models, moral reasoning, causal intervention, circuit discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_G7yvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how LLMs internally represent event relations and use them to make moral judgments. The authors propose ENACT, a three-stage causal framework, and introduce MoralTrace, a synthetic diagnostic dataset for controlled counterfactual interventions. They analyze moral-patient representations, causal attribution, and relational binding across different models, followed by circuit-level analysis and generalization experiments.

Critical strengths

The paper includes extensive experiments across 27 models from 11 model families, using controlled counterfactual prompts, causal interventions on internal representations, and circuit-level analyses to investigate the mechanisms underlying moral judgments in LLMs.

Critical weaknesses

The paper is difficult to follow, starting from the introduction. The authors do not provide a self-contained introduction to the Theory of Dyadic Morality. The proposed three sequential causal transitions are also not sufficiently motivated. Simply referring to prior work is not enough, the paper should provide a clearer justification for why this particular causal structure is appropriate. It is difficult to understand how the experiments in Sections 3–5 support the proposed causal framework, particularly how they establish the sequential dependencies between the individual components.

Questions for authors
  1. How were the representative models in Table 2 selected? The full results in Table 8 seems more mixed, so it is helpful to discuss the selection process and the overall consistency of the findings across models.

  2. The proposed MoralTrace dataset was generated and validatied by an LLM with manual review on examples flagged by the validating LLM. It would be helpful to see some quantitative results for the LLM-based and maual validation. How many examples were flagged? How many of the flagged examples were excluded? Is there any manual check on examples that pass LLM validatin?

  3. How does the proposed ENACT framework for moral reasoning connect to more general causal or relational reasoning?

  4. How is rank 4 chosed for distributed alignment search?

3 comments
AuthorsOpenReview ↗

Response to Reviewer G7yv

Thank you for pointing out that the original paper introduced TDM and the three operators without enough explanation of how the experiments test them. We revised §2.1 to explain the theory and state the contrast for each operator, and added an intervention in §3.3 that tests whether patient completion affects attributed intention.

Why these operators, and what depends on what? TDM treats perceived harm as a relation between an agent and a vulnerable patient, with norm violation and negative affect also contributing to condemnation. We use this relation to construct independent contrasts. In the office example illustrated in Figure 2b, Rachel opens a window, Ethan’s report pages scatter, and he misses a deadline. In §3, leaving the affected person implicit tests whether the model represents Ethan as the patient. In §4.1, a gust from another, already-open window changes who caused the outcome while holding the missed deadline fixed. Crossing source with valence tests whether the outcome changes intention primarily for the causal agent. In §4.4, a duty to secure the pages illustrates how the same event can be judged under different rules. The criterion-replacement experiment implements that contrast in an allocation task. Section 5.1 traces the source–valence update through attention and MLP outputs to the late subspace.

Adding the rank-two completion component changed intention ratings by +1.28 points in Llama-3.1-8B, +1.77 in Gemma-2-9B, and +0.03 in Qwen2.5-14B. Removing it lowered wrongness in all three, even when the written intention answer was fixed (§3.3, Appendix B.2, lines 1551–1565). These tests show that patient information can affect intention and can also affect wrongness when the explicit intention answer is held fixed. A single obligatory sequence through an internal intention state remains unestablished.

Q1. Model selection and consistency. Table 1 shows examples from several model families and three Qwen2.5 sizes, allowing comparison across families and within one family. We now report the aggregate pattern in §3.3 and the full results in Table 11 (Table 8 in the submission). Counts below use model-level pointwise 95% bootstrap intervals excluding zero.

Wrongness responseRank 1Rank 2At either rank
Increased after addition172122
Decreased after removal111216

Fifteen of 27 models showed both predicted responses when counting each intervention across ranks. At rank two, removal increased wrongness in five models and its interval spanned zero in ten. Addition was more consistent than removal.

Q2. Validation counts. The authors reviewed the hand-written templates. The generation model was also used in a separate temperature-zero semantic check for unintended cues about intention, damage, or emotional reactions. Appendix D.6 reports the available generation logs.

Generation suiteAttempts with automated errors
Single-chain, train/validation9 / 308
Converging causes, train/validation20 / 356
Relation reassignment103 / 703

The denominators count generation attempts. The archived records do not establish how many items the semantic check flagged, how many were excluded after manual review, or whether passing items were manually spot-checked.

Q3. Relation to general causal reasoning. The same causal relation can inform a judgment of responsibility for a loss or an assessment of whether an operation caused a system failure. We hold the event graph fixed across moral, coordination, and technical tasks. Coordination retains the interpersonal setting while changing what counts as a poor action. In §5.2, transferring the activation difference induced by reassigning a cause recovered 56%–109% of the full-state intention shift across 16 cross-domain transfers in three models. This supports reuse of a causal update across these tasks. The criterion intervention (§4.4, Appendix B.9) separately tests how the applicable standard influences the final judgment.

Q4. Rank four in DAS. Four was a fixed subspace size for comparing patient, intention, and outcome-severity interventions with their same-rank random controls. We varied the layer and kept rank and training settings fixed (§3.1, Appendix A.4, lines 960–971). We did not run a rank sweep for this experiment.

AuthorsOpenReview ↗

Response to Reviewer G7yv

Dear Reviewer G7yv,

Thank you for pointing out that the connection between TDM, the operators, and the experiments was difficult to follow. In preparing the original submission, we spent much of our time organizing the experimental results and underestimated how much explanation readers would need to see why each test followed from the framework. We learned a lot from going back through the paper with that question in mind.

In the final revision, we moved Figure 1 into the Introduction, rewrote parts of §§2–3 to explain how the framework leads to the experimental comparisons, and added Table 1 to show how the explicit-patient, implicit-patient, and matched control conditions are constructed. We also retitled §3.3 Causal Contribution of Patient Completion and explain alongside the results that removing the patient component can lower wrongness ratings even when the written intention answer is held fixed.

In §4.2, we replaced the earlier joint-effect table with the token-level plot previously in the appendix (now Figure 5) and revised the text and caption to explain what the plot tests. We moved the takeaways to the start of §§3–5 and reorganized the appendices so each research question points to its methods, controls, and full results.

We appreciate your careful reading. Your questions helped us explain how the experiments bear on the framework one step at a time.

Thank you for the detailed response. The explanations help clarify how the individual experiments connect to the proposed framework.

The revised presentation addresses some of my concerns, but I'm still not fully convinced that the evidence supports the broader claims about the proposed mechanism. I will maintain my score.

Reviewer_ZBYZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates how large language models compute moral judgments, proposing that they operate through three core steps derived from the Theory of Dyadic Morality (TDM): completing an implicit moral patient, inferring intention from valence and causality, and applying a normative criterion. To test this, the authors introduce MoralTrace, a synthetic corpus sharing event graph structures across mechanistic, instrumental, and normative-social domains. Using activation patching, domain-adversarial subspace training (DAS), circuit search, and SAEs across up to 27 models, they conclude that models encode implicit patients affecting wrongness ratings, combine source and valence non-additively, and localize this computation in a roughly one-dimensional late-layer subspace that transfers across domains and partially to datasets like Moral Stories, MoCa, AITA, Scruples, and UniMoral.

Critical strengths

The empirical work employs causal intervention tools, such as random, shuffled, and wrong-graph controls, rather than relying solely on passive probing. The minimal-pair design is carefully controlled for length, template-level splits, and order. Furthermore, distinguishing between a directional shift in output versus structural routing through the event relation offers a valuable perspective for the representation engineering and steering literature, even though the formalization in Appendix C is mostly surface-level.

Critical weaknesses

First, the theoretical framework relies heavily on a contested model of morality using stimuli that lack human validation.

TDM is treated as the definitive formal structure of moral cognition, bypassing active debates surrounding moral pluralism and alternative frameworks (e.g., Graham et al., 2013; Rai & Fiske, 2011; Curry et al., 2019). Human moral judgment often penalizes harmless purity or authority violations where no clear victim is present (Haidt et al., 1993; Young & Saxe, 2011), yet MoralTrace contains no such items. Moreover, the formulation in Appendix C essentially treats any act without a negative outcome as benign by construction.

The assumed feed-forward order (patient →\rightarrow intention →\rightarrow judgment) lacks strong backing in cognitive science. TDM itself posits that dyadic completion flows backward from perceived wrongness to perceived suffering (Gray et al., 2014), and extensive literature shows that moral evaluation actively shapes mental-state attributions (Alicke, 2000; Cushman, 2008; Malle et al., 2014). In fact, the Knobe effect examined in Section 4.1 is classic evidence of evaluation influencing intention ascriptions.

The paper justifies synthetic stimuli by claiming that "moral judgment lacks a clean external ground truth", yet 1. this is one of the domains in behavioral sciences where large amounts of empirical data (i.e., human "moral judgments") exists, and 2. large-scale human annotated/validated datasets, such as Moral Machine, Scruples, MoCa, Social Chemistry, and moral foundations corpora, are readily available. No human ratings were collected for MoralTrace items, and the text-generation model itself was used to validate the dataset (App. D.6). It remains unclear whether humans view the normative-social items (such as abstract resource allocations or scattered papers) as moral domains, or whether they exhibit the specific asymmetries targeted by the model readouts. Finding that the same subspace drives evaluations of cache controllers and documentation reviewers points more toward generic causal attribution than specifically moral reasoning.

The Knobe effect analysis has additional limitations: While the side-effect asymmetry replicates (Cova et al., 2021), the effect size drops significantly when participants are given a knowingly option instead of a binary intentionally choice (Guglielmo & Malle, 2010). The effect is often driven primarily by general norm violations rather than harm per se (Uttich & Lombrozo, 2010). The Teresa vignette closely paraphrases Knobe’s original CEO case, which is almost certainly part of the models' pretraining data.

Prior mechanistic work on moral representations is only cursorily addressed, omitting key findings that sanctity dimensions occupy distinct representational axes in many architectures (e.g., Yu et al., 2026; Karami et al., 2025). Additionally, datasets like Moral Stories and MoCa are used without proper citation.

Several central empirical claims outrun the evidence presented in the paper.

The claim of spontaneous completion (RQ1) relies on prompts where the patient is explicitly introduced via a possessive noun phrase (Felix's cherished sculpture). Furthermore, the contrast pairs pair the patient with destructive verbs (shattered vs. moved), meaning the extracted directions largely capture harmful actions rather than patient presence alone. LEACE erasure fails to eliminate the rating gap, and significant patient-removal effects appear in only 11 to 12 out of 27 models (App. B.2; conflicting with the main text's claim of 16), with some models moving in the opposite direction.

The "alternative valence shortcut" is never defined. In Table 13, baseline valence models recover .86–.94 of the effect for Qwen2.5-7B, compared to .94–1.12 for the joint direction, suggesting a very narrow margin.

Qwen2.5-7B violates the paper's own inclusion threshold (showing an order sensitivity of .48 against a .10 bound) yet drives a substantial portion of the main text. Meanwhile, Llama-3.1-70B, a core model where the joint interaction effect is practically zero (Δjoint≈0\Delta_{\text{joint}} \approx 0), is omitted from Table 3.

A positive joint interaction (Δjoint>0\Delta_{\text{joint}} > 0) on bounded ratings or logit margins can easily stem from downstream saturation effects without any actual feature binding occurring at the patched layer. Notably, the outcome-only effect is negative across every single row in Table 9.

The rank-1 subspace is explicitly optimized to fit the scalar effect it is subsequently claimed to recover. Using shuffled supervision already achieves an R-squared of .37 compared to .93.

At K=64K = 64, using an incorrect, randomized circuit mask recovers nearly as much performance as the true discovered circuit. Inconsistent reporting: The chosen layer indices for the frozen subspace vary across Appendix B.8, Tables 12–13, and Figure 11. Furthermore, the cited Gemma Scope SAE (for the instruction-tuned model at layer 32) does not correspond to any publicly released layers (9, 20, 31).

The evaluation against human judgments shows weak alignment. On Scruples, the extracted direction correlates at r=.11r = .11 and .17.17 with human wrongness ratings, failing to beat random directional controls (Table 18). On Moral Stories, ablation removes 18–38% of the effect, but two out of seven models exhibit the opposite sign. I also thought the author don't believe in human moral judgments ("moral judgments provides no single external ground truth")...

Questions for authors

Do human evaluators rate the normative-social MoralTrace items as genuinely moral, and do their empirical ratings exhibit the harm/help and agent/background asymmetries assumed by the model readouts?

Is the proposed operator sequence tested directly via multi-step causal mediation? For instance, does intervening on the patient completion state reliably alter the downstream intention state, and can the analysis detect feedback loops from judgment back to intention attributions?

Do the core findings hold when restricting analysis strictly to the pre-registered model suite? Specifically, why is Qwen2.5-7B included despite failing the order sensitivity admission screen, and why is Llama-3.1-70B omitted from Table 3?

Does the positive joint interaction metric survive a linear rescaling of the readout scale? Why is the outcome-only ablation effect negative across all rows in Table 9?

How do the side-effect results shift when models are given a knowingly response option or tested on novel vignettes outside the standard Knobe (2003) formulation?

Could the authors clarify exact implementation details: which specific layer defines the frozen subspace for each model, and which Gemma-2-9B SAE release and layer index were utilized?

Clarification: do the metrics in Table 1 reflect generation acceptability or wrongness ratings (Sec. 3.3, Table 2)? Where can readers find the pre-registration documentation for the stated criteria?

How do you propose your model can capture cross-cultural variances in moral judgment?

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZBYZ (1/3)

Thank you for the detailed review. Your questions about the ordering of the operators and the origin of the joint effect prompted new intervention tests. We also revised the explanation of TDM and MoralTrace (§§2.1–2.3) and clarified what the external evaluations establish.

Part 1 of 3 | Theoretical scope, operator dependencies, and side effects

TDM and moral pluralism. We use TDM to specify manipulable event relations, including who bears a consequence and who caused it. Section 2.1 now explains the roles of perceived harm, norm violation, and negative affect. Appendix C.1 (lines 2047–2051) clarifies that a zero target in the absence of an adverse outcome means zero fault for that outcome. Moral evaluations based on purity or authority fall outside that target. MoralTrace’s B5 subset contains a limited set of purity and taboo scenarios used to probe completion when a patient cannot be inferred (Fig. 8, Appendix C.1). We do not claim coverage of all moral concerns.

Operator dependencies and feedback (Q2). The new tests support an effect of patient completion on attributed intention in two of three models. In a two-question prompt, removing the same component still lowered wrongness with the written intention answer held fixed (§3.3, Appendix B.2, lines 1551–1565).

ModelIntention change after additionWrongness change after removal, free intentionWrongness change after removal, fixed intention
Llama-3.1-8B+1.28−0.23−0.21
Gemma-2-9B+1.77−3.28−3.32
Qwen2.5-14B+0.03−1.81−2.01

Values are rating points. The intention column uses a separate, single-question readout. The wrongness columns use the two-question prompt. These interventions leave open whether an internal intention state mediates the effect on a standalone wrongness question. In the reverse direction, increasing the written wrongness rating by one point raised subsequent intention ratings by 0.06–0.26 points across models and vignette sets, exceeding the description-length control in all six comparisons. Fixing the written wrongness answer left 57%–75% of the original harm–help gap. With that answer fixed, swapping states at the wrongness question yielded no consistent transfer on the side-effect vignettes (Appendix A.6.2, lines 1188–1199). The feedback result concerns an explicit earlier answer and does not establish a recurrent internal pathway.

Response options and new vignettes (Q5). We added a “knowingly but not intentionally” option to the same 389 harm–help pairs and tested 48 new pairs covering routine actions, choices, and indirect consequences. The new pairs hold the action, goal, knowledge, attitude, and rule compliance fixed within each pair (Appendices A.6.1–A.6.2, Tables 4–5).

ModelTwo options (pp)Three options (pp)New pairs, 1–7 rating gap
Llama-3.1-8B7.954.69+0.153 ± 0.072
Gemma-2-9B33.5622.32+0.527 ± 0.232
Qwen2.5-7B0.234.07+0.566 ± 0.153

The first two columns report the harm–help gap in the probability of “intentionally,” in percentage points averaged across option orders. Qwen2.5-7B was near floor with two options (below 0.4% in each condition). Scoring by the highest-probability option gives the same direction of change for Gemma and Qwen, with an interval spanning zero for Llama. The new-pair gaps are all smaller than in the original scenarios and their pooled 95% intervals exclude zero (± gives the half-width). For Llama and Gemma, the interval spans zero in the choice-between-actions subset. These results show sensitivity to response format and extend the effect beyond the original CEO-style scenarios. They leave open how strongly norm violations modulate the asymmetry.

Cross-cultural variation (Q8). Both the norm applied and the perceived harm can vary across cultures. For example, damaging a ritual object may be seen as harming a community in one setting and as damage to property in another. The criterion intervention holds event facts fixed and changes the applicable allocation rule, shifting judgments by 0.51–1.61 logits across four models (§4.4, Appendix B.9). That establishes sensitivity to stated norms in our tasks. We did not test differences between cultural populations.

AuthorsOpenReview ↗

Response to Reviewer ZBYZ (2/3)

Part 2 of 3 | Patient representations, model inclusion, and joint replacement

Patient completion versus damage. In the Felix example, the owner is named and his role as the affected party is inferred. The original B–D contrast also changes the action from moving to breaking the sculpture. In the ownerless-damage control, the median projection relative to owned damage was 65% across B1–B4, and 14% for off-scene individuals (26/27 pointwise 90% intervals below one). In symbolic cases it remained 89% (Appendix A.5, lines 1039–1046). We then held the damaging sentence fixed in 48 new templates and varied which object the named person owned or relied on. The patient-direction projection tracked that relation in all three models. Removing the direction at the scene boundary left the wrongness gap essentially unchanged in Gemma-2-9B and Llama-3.1-8B and widened it by 0.20 points in Qwen2.5-14B (Appendix A.5.1, Table 3). The direction is sensitive to the affected-party relation as well as damage. The fixed-damage intervention does not show that the direction is necessary for the rating gap.

LEACE and removal counts. Erasing information that distinguishes B from D nearly eliminates their wrongness gap in five models. Erasing information defined by patient presence (A/B versus C/D) leaves a gap (Appendix A.5). In the full completion study, removal lowers ratings in 11 models at rank one and 12 at rank two, with seven in both groups and 16 in their union. At rank two, removal raises ratings in five models and the interval spans zero in ten. Addition raises ratings at either rank in 22/27 models, and 15 show both predicted responses (§3.3, Table 11). Counts use pointwise 95% bootstrap intervals excluding zero.

Screening and Llama-3.1-70B (Q3). Qwen2.5-7B is outside the core: its 48.1-point label-order sensitivity exceeds the 10-point bound, and its implicit patient accuracy is 69.1%. Its 92.8% causal-source accuracy supports narrower, within-prompt source and joint replacements comparing matched with wrong-graph donors (Table 9, Appendix D.8). Those results do not establish order robustness. Table 2 summarizes selected models and omits this weaker case. Revised Table 12 explicitly reports both layers for all four core models. For the four core models, seven of eight model–layer intervals for the joint excess lie above zero. Llama-3.1-70B is the weakest case:

Llama-3.1-70B layerJoint excess ± 95% bootstrap half-width
24+0.05 ± 0.04
320.00 ± 0.02

The second interval spans zero. The joint excess is thus present at most tested core layers, with a small or negligible effect in Llama-3.1-70B.

Outcome column, rescaling, and downstream nonlinearity (Q4). The Outcome column in submitted Table 9 (revised Table 12) reports the control-subtracted difference in valence transfer between agent and background causes. A negative entry means smaller transfer under an agent cause. Both Llama-3.1-70B entries are +0.01 at the reported precision. In an additional reanalysis of the 38 common held-out templates, the outcome-only effect from a beneficial background outcome ranged from −0.525 to +0.678 logits across 14 model–layer settings. For the sign-aligned joint contrast averaged across all four starting conditions, six pointwise 95% paired bootstrap intervals excluded zero (one positive, five negative). A common affine rescaling r′ = ar + b with a > 0 multiplies the interaction by a and preserves its sign. It can still reflect downstream nonlinearity.

We tested this directly by writing the observed joint state or the sum of the two single-factor state changes plus the original state at the same token and layer, then passing each through the same downstream computation (Appendix B.3.1, Table 13).

Model and layerJoint state (logits)Downstream (logits)Joint-state share
Gemma-2-9B L13−0.033 ± 0.011−0.017 ± 0.01566%
Gemma-2-27B L18−0.111 ± 0.038−0.042 ± 0.02672%
Qwen2.5-72B L24−0.032 ± 0.022−0.142 ± 0.03318%
Qwen2.5-72B L32−0.009 ± 0.031−0.207 ± 0.0494%

Values are signed contributions to the intention interaction (± 95% confidence half-width). The joint state contributes most at these Gemma layers, whereas downstream processing of the additive state contributes most at both Qwen2.5-72B layers.

AuthorsOpenReview ↗

Response to Reviewer ZBYZ (3/3)

Part 3 of 3 | Human validation, external tests, and implementation

Human validation and dataset construction (Q1). We did not collect item-level human ratings for MoralTrace. Whether people treat the same scenarios as moral and exhibit the harm/help and agent/background asymmetries remains open. Scruples and Moral Stories informed the choice of everyday moral prototypes. Hand-written graph templates specify who is affected, who caused the outcome, and what responsibilities apply before generating surface variants (§2.3, Appendices A.3.2 and D.4). The generation model was reused in a separate temperature-zero semantic check for unintended cues about intention, damage, and emotional reactions. Appendix D.6 reports automated errors in 9/308 single-chain, 20/356 converging-cause, and 103/703 relation-reassignment generation attempts. The logs do not establish semantic flag counts, exclusions after manual review, or spot checks of passing items.

External results and scope. The Scruples projection gives weak human-rating correlations (Table 22). On Moral Stories, the two negative entries are for intention (Qwen2.5-14B and Gemma-2-27B). The wrongness interaction is positive in all seven models (Table 18). Ablating the frozen subspace removes 18.5%–37.8% of the wrongness interaction in the three models tested (Table 19). These interventions measure changes in model judgments. Human ratings provide a separate measure of alignment.

Scruples modelPearson r with human ratingsEmpirical p vs. random directions
Qwen2.5-7B0.1070.296
Gemma-2-9B0.1680.275

Cross-domain transfer (§5.2) supports your interpretation that the causal update is shared with more general relational reasoning. The moral application concerns how the update informs the evaluation of an action under a norm, as tested separately in §4.4. Section 6 now discusses Yu et al. (2026) on care and sanctity directions and Karami et al. (2025) on decodable moral foundations. Section 5.2 cites Moral Stories (Emelin et al., 2021) and MoCa (Nie et al., 2023).

Global valence, rank-one recovery, and mask controls. A global valence shortcut predicts that the same agent’s evaluation should stay similar when we swap who caused the harmful and beneficial outcomes while keeping both outcomes fixed. The role-reassignment experiment finds that attributed intention changes with this swap (§4.3). For Qwen2.5-7B, the valence-only direction nevertheless produces most of the intention shift of the joint direction, with 0.856–0.941 logits for valence alone and an additional 0.079–0.182 logits for the joint direction (Table 17).

The rank-one direction is fit to the full joint replacement effect and tested on disjoint graph templates. The 0.37 cited in the review is a logit shift under shuffled training, and 0.93 is a recovery ratio under matched training. For a comparison in the same units and setting, mechanistic-domain Qwen2.5-7B shifts by 0.925 logits with matched supervision and 0.369 with shuffled supervision (Table 16).

In the Qwen2.5-7B normative setting, the wrong-graph and matched joint masks recover −2.0% and 10.9% of the intention effect at K = 8, versus 87.2% and 85.2% at K = 64. Equal-size random sets recover 13.1%–38.9% across settings at K = 64, averaged over two draws per setting. An optimized wrong-graph mask is a stronger control than random sampling, and recovery alone offers little evidence of relation specificity at the larger budget (Appendix A.7, Table 6).

Implementation and rating scale (Q6–Q7). Appendix B.8 now lists the layers together. Tables 16–17 give the internal validation layers, Fig. 12 labels the circuit readout, and the last column gives the frozen layer used for external transfer.

ModelTable 16Table 17Fig. 12External transfer
Gemma-2-9B37373728
Gemma-2-27B30302324
Qwen2.5-7B28282424
Qwen2.5-72B6161—61
Llama-3.1-8B2524—24
Llama-3.1-70B6464—64
OLMo-3-7B3232—32

For the Gemma-2-9B feature mediation, the SAELens release is gemma-scope-9b-it-res-canonical, checkpoint layer_31/width_16k/canonical. Its zero-based layer 31 corresponds to block 32 in the paper’s one-based numbering (Appendix A.7, lines 1275–1278).

We removed the term “preregistered.” Appendix D.8 gives the model admission criteria and identifies the validation reports recording thresholds and decisions. Appendix A.8 gives the separate dose–response criteria for steering. Section 2.3 (lines 201–202) describes the moral acceptability scale, from 1 (completely acceptable) to 7 (completely unacceptable). Section 3.3 and Table 1 call this the wrongness rating and report changes on that same scale.

Reviewer_b33qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors study how LLMs encode information about moral judgments. Relying on the Theory of Dyadic Morality, they build MoralTrace, a dataset containing counterfactual scenarios used to evaluate the analyzed properties. They then employ various mechanistic interpretability techniques (e.g., activation patching, SAEs) to understand how the components of moral judgments are encoded in models and whether models internally implement the ENACT (Event-Navigated Causal Transition) process: first forming event relationships and then using them to predict the outcome. Finally, they show that models exhibiting such behaviors generalize them to other topics sharing the same underlying causal structure.

Critical strengths
  • The analysis and experiments are extensive, covering diverse scenarios and multiple models.
  • The proposed dataset is interesting and could be valuable for other areas (e.g., faithfulness), as it provides counterfactual pairs.
  • The distinction between an activation being steerable and actually being used by the model (what they call the “steering-routing gap”) is an insightful and relevant point.
  • The experiments are well thought out, incorporating numerous controls and analysing different factors.
Critical weaknesses
  • Sections 2.1, 2.2, and 2.3 are critical for understanding the paper, yet they are very difficult to follow on a first read. It would be helpful to include concrete examples grounding these concepts in actual model components or realistic scenarios, similar to what Table 1 does for Section 2.3.

More broadly, the paper is hard to navigate. Crucial details often appear only in the appendix, and the manuscript lacks a high-level overview explaining what is being tested and why.

  • For instance, MoralTrace is introduced in just a few lines in Section 2.3, leaving the design choices in Table 1 largely unmotivated.

  • Similarly, Section 3.1 refers to "three target models" and "held-out layer selection" without specifying which models are used or how the layers are held out. I only found these clarifications in the appendix.

  • In the main text, it is frequently unclear what a given experiment is testing, what the core hypothesis is, and what outcome would support or falsify it. While this context might exist across the 43 pages, a reader should not have to cross-reference the appendix for every sentence. The takeaway boxes are helpful, but they cannot compensate for a main text that is not self-contained. The results are clearly substantial, but condensing too much content into the main body creates chaos and leaves insufficient context to interpret the findings without digging into supplementary material.

Questions for authors

See above.

  • Could you include a high-level running example throughout the paper that walks through each step, hypothesis, and experiment, connecting prompt examples to internal model components?

  • Is the Route metric evaluated empirically anywhere? Since it is one of the central motivations for the paper, directly measuring it would significantly strengthen the work.

  • How were the specific models chosen for each set of experiments?

3 comments
AuthorsOpenReview ↗

Response to Reviewer b33q

Thank you for pointing out where the main text was difficult to follow and for suggesting a direct test of routing. We revised §§2.1–2.3 to explain the event variables, why MoralTrace varies them independently, and what would count as routing. We also moved the model and layer selection details into §3.1 and added a direct measurement of Eq. (3) in §4.3.

Running example and the purpose of each test. In the office vignette illustrated in Figure 2b, Rachel opens a window to air the room, pages from Ethan’s report scatter, and Ethan misses a deadline. The dyad identifies who bears the consequence (Ethan), while the event graph distinguishes whose action we evaluate (Rachel) from what actually caused the outcome.

  • Patient completion (§§3.2–3.3). If the story leaves the affected person implicit but retains the ownership relation, the model can still infer that the missed deadline affects Ethan. We test whether the scene’s residual state has a component in common with the state elicited by an explicit patient, and whether changing that component shifts wrongness ratings. A new control keeps the damage identical and varies which object the person owns or relies on (Appendix A.5.1, Table 3).
  • Causal attribution and judgment (§§4.1–4.4). As illustrated in §4.1, suppose the pages were scattered by a gust from another window that was already open. Rachel’s action and the missed deadline stay the same. Crossing these causal variants with harmful and beneficial outcomes tests whether valence has a larger effect on attributed intention when Rachel caused the outcome. A separate contrast varies Rachel’s duty to secure the papers while holding the event fixed, illustrating the criterion intervention in §4.4.
  • Components (§5.1). We use matched prompts that differ in causal source and valence to identify which attention-head and MLP outputs carry the change in attributed intention. We then test how those component outputs contribute to the late low-dimensional state (Fig. 6 and Appendix A.7, Table 7).

Direct measurement of Route(w). On 78 held-out role-reassignment templates, we crossed which agent was queried with which agent caused the harm. We applied the same fixed valence direction in all four cells and measured the change in the Yes–No logit margin per unit of residual norm. Double centering removes shifts attributable to the queried agent alone, the cause alone, or the story as a whole. The signed interaction (RRI) gives the direction of the effect, and its magnitude equals Route(w) in this two-agent design (§4.3, lines 389–393, and Appendix A.8.1, Table 8).

ModelReadoutMean absolute responseRoute(w)Signed RRI ± 95% half-width
Qwen2.5-14BIntention0.08340.0640+0.0640 ± 0.0059
Qwen2.5-14BJudgment0.09430.0016−0.0016 ± 0.0148
Gemma-2-9BIntention0.01660.0207+0.0207 ± 0.0014
Gemma-2-9BJudgment0.00610.0123+0.0123 ± 0.0010

For intention, both models showed a larger response for the causal agent, and their signed RRI exceeded all three fixed random-direction estimates. Qwen2.5-14B’s judgment response was larger on average than its intention response, but positive and negative template-level interactions nearly canceled when pooled.

Models and layer selection. For distributed alignment search (§3.1, Appendix A.4, lines 960–971), we used similarly sized models from three families (Llama-3.1-8B, Qwen2.5-7B, and Gemma-2-9B). At each of six layers, we fit on the same half of the counterfactual pairs and evaluated on the other half. We selected the layer with the largest held-out gain in interchange accuracy over a same-rank random subspace. For the separate patient-direction analysis, we selected the earliest layer whose five-fold mean held-out Cohen’s d for the explicit-patient contrast was within one standard error of the peak. We then used that layer to test implicit patients.

The patient-completion study covered 27 models to assess how widespread the effect is (Table 11). For the main mechanistic analyses, four of 17 formally screened models passed the single-chain criteria (Table 9, Appendix D.8): Gemma-2-9B, Gemma-2-27B, Qwen2.5-72B, and Llama-3.1-70B. Each needed at least 75% accuracy on explicit-patient, implicit-patient, and causal-source identification, measurable valence effects on intention and judgment, a Yes rate between 5% and 95%, and at most a 10-percentage-point shift under label or clause reordering. These checks establish that the model can identify the manipulated roles and has a behavioral effect to trace.

I thank the authors for the additional clarification and the updated PDF. I believe the paper is in a better shape now, but I am still a bit concerned about the readability and how the paper (especially the results) is presented, which makes it a very dense and messy set of results. I will keep my scores.

AuthorsOpenReview ↗

Response to Reviewer b33q

Dear Reviewer b33q,

Thank you for reading the first revision and telling us where the results were still hard to follow. Before submission, much of our time went into completing and checking the experiments. Your comments made us realize how much work we had left to the reader to connect each result to the question it was meant to answer.

In the final revision, we moved Figure 1 into the Introduction, use Figure 2 to walk through the scenario design, and introduce the four patient-completion conditions in the new Table 1 before presenting the intervention results. We also rewrote §4.2, Separate Encoding of Causal Source and Outcome. The earlier joint-effect table there has been replaced by a plot brought forward from the appendix (Figure 5), so readers can see which token positions carry each variable. We revised the surrounding paragraphs and captions to explain how the comparisons support the conclusions.

We also revised the subsection headings, moved the takeaway boxes to the start of the corresponding results sections, and added a roadmap to the reorganized appendices. Your feedback helped us see the manuscript from a first-time reader’s perspective.

We would be grateful if you could consider these changes in your final assessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Thank you for the careful reviews. We revised the manuscript to make the experiments easier to follow and added two tests in response to your questions.

  • Running example (§§2–4, Figure 2b). We use the office scenario to show how the experiments distinguish who was affected, who caused the outcome, and which norm applies.
  • Direct test of routing (§4.3, lines 389–393, and Appendix A.8.1, Table 8). On 78 held-out templates, the same valence intervention shifted attributed intention more for the agent who caused the harm in both tested models.
  • Dependencies between operators (§3.3, lines 293–296, and Appendix B.2). Adding the patient-completion component changed attributed intention in two of three models. Removing it lowered wrongness ratings in all three even when the written intention answer was held fixed. Whether an internal intention state mediates this effect remains open.

We respond to each reviewer in the corresponding thread.

AuthorsOpenReview ↗

General Response — Final Revised Manuscript

Dear Area Chair and Reviewers,

Thank you for the careful reviews and follow-up discussion. We have uploaded our final revised manuscript. The discussion helped us see where readers had to reconstruct the logic connecting the framework, experiments, and conclusions. We used the revision period to work through those sections paragraph by paragraph, including the results descriptions, headings, captions, and supporting analyses.

  • Introduction and experimental design. We moved Figure 1 from §2.1 into the Introduction so readers can see how our causal approach relates to prior steering work when we first introduce it. Figure 2 visualizes the scenario construction originally presented as a table, and the new Table 1 gives concrete examples of the four patient-completion conditions and their prompts before the results.
  • Results and takeaways. In §4.2, we replaced the joint-effect summary table from the first revision with a token-level plot previously in the appendix (now Figure 5). We rewrote the subsection and its caption around what the plot shows about where causal source and outcome information are represented. We also revised subsection headings to name the questions being tested and moved the takeaway boxes to the start of §§3–5, where they orient readers before the detailed analyses.
  • Appendices. We consolidated repeated material, grouped related controls and results, and added a roadmap from each research question to its methods and full results. For example, readers can follow the token-level evidence in §4.2 to Appendix B.3, where Table 11 quantifies the contributions of the joint activation and downstream processing.

We learned a great deal from the discussion and appreciate the time you took to show us where the manuscript needed more work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.