- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces MoralLedger, which is a benchmark that studies whether an LLM's past moral behavior affects its decisions in unrelated situations. Using around 1k examples and eight models, the authors find that positive moral histories have a stronger effect on future decisions than negative histories. They also identify a pattern in the models' internal representations that is linked to moral choices and works on unseen examples. Lastly, they show that changing model activations using this pattern can influence moral decisions in both directions (more effectively than prompting alone) without much impact on general performance.
Critical strengths
- The paper has a well-designed experimental setup that separates past moral behavior from the current moral decision.
- The paper studies the same question from three different pov. The additional experiments and checks make the main findings more convincing.
- The paper includes several additional experiments to test how reliable the findings are which help rule out alternative explanations for the results.
- The authors clearly reports differences across models, including cases where the effects are weak or inconsistent.
Critical weaknesses
- The decision scenarios are generated and validated by LLMs, without human evaluation. Since the benchmark relies on these scenarios having a clear trade-off between morality and self-interest, human validation on a subset would make the results more reliable.
- The steering experiments only measure probabilities for three fixed answer choices, rather than actual generated responses. Testing whether steering also affects open-ended responses would make the findings more useful for real-world applications.
- The effects vary considerably across models, but the paper does not explain why some models are more affected than others. Further analysis of model architecture or training differences could help explain these variations.
- The non-moral control direction is estimated using a different dataset, which may introduce differences beyond moral content. Using moral and non-moral examples from the same dataset would make the comparison more reliable.
- The paper claims that activation steering is stronger than prompting, but this is not consistently supported across all models. The authors should make this claim more carefully, especially in the abstract.
- The benchmark uses English scenarios that mostly reflect Western moral values and focus on money or time-related decisions. The paper should discuss how these choices might limit the findings across different cultures and settings.
Questions for authors
- How sensitive are the RQ1 results to the wording of neutral histories? Could these histories still carry some positive or negative meaning that affects the comparison?
- Why do some models, like OLMo 2 1B, show moral information in their internal representations but little change in their decisions? Could this be related to how different model layers use this information?
- Could the identified moral direction simply reflect sentiment or specific words associated with morality, rather than moral reasoning itself? Have the authors tested this beyond the non-moral control?