- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces Synth-Ethics, a synthetic dataset of 68,520 reasoning traces covering 4,397 moral dilemmas and 16 ethical frameworks. Each framework is operationalized through an expert-authored framework card that guides the generation of framework specific reasoning and verdicts. The authors fine-tune nine base models from three families and use MoReBench to evaluate framework agreement, ood behavior, and reasoning adherence. They find that fine-tuning aligns models with the target framework more closely than prompting does, and that fine-tuned models exhibit framework specific reasoning without explicit instructions. The authors also use the dataset to profile the moral judgments of 27 API models and find a general tendency toward virtue ethics.
Critical strengths
-
The paper moves beyond crowd sourced moral judgments by constructing parallel reasoning traces for 16 explicit ethical frameworks. The expert authored framework cards and controlled causal structure provide a systematic foundation for studying differences in moral judgments. This is a meaningful contribution to pluralistic alignment and moral evaluation.
-
The evaluation spans nine base models, 144 single framework fine-tunes, and independent benchmarks. The MoReBench results are particularly interesting: fine-tuned models consistently score highest on their target framework's rubric even without explicit framework instructions. This provides evidence that framework-specific behavior can be learned through training rather than induced solely through prompting.
Critical weaknesses
-
The same LLM-generated framework resolutions serve as both training targets and evaluation references. Although the authors conduct human evaluation on 800 traces, this primarily assesses whether the prescribed reasoning procedures are followed, rather than independently validating the moral verdicts. The ratings are also heavily concentrated near the ceiling (Appendix C). Consequently, agreement with these references may reflect successful imitation of the generator's interpretation of each framework rather than genuine adherence to the underlying ethical theory. This limitation also affects the validity of the conclusions drawn about existing API models.
-
Unclear responses are excluded from agreement calculations, despite substantial differences in answer coverage across models (Table 6). This makes the fine-tuning versus prompting comparison potentially sensitive to selective nonresponse.
-
The out-of-corpus evaluation reports overall action selection rates rather than independently validated framework specific judgments. Differences in these rates could reflect general response biases rather than ethical reasoning.
Questions for authors
-
Can the authors provide independent expert validation of the reference verdicts, particularly on scenarios where the ethical frameworks disagree? How frequently do experts agree with the generated judgments, and how sensitive are the main results to disagreements?
-
What proportion of held-out scenarios produces judgeable verdicts for each fine-tuned and prompted model? Does the fine-tuning advantage persist when Cohen's κ is computed on the same subset of scenarios or when abstentions are explicitly accounted for?
-
Can the authors provide stronger ood evidence of framework adherence, such as testing whether models respond appropriately to controlled changes in morally relevant causal factors? The current actionnselection rates alone do not establish that the learned behavior follows the intended ethical principles.
2 comments
W3 and Q3. External datasets that judge dilemmas within named ethical frameworks are scarce: in our survey of moral-reasoning benchmarks, MoReBench is the only public dataset that scores a model's reasoning against expert criteria written for each theory. There, with no theory named, each of the 36 fine-tunes for the four frameworks it covers scores highest on its own framework's rubric (Table 11), the result you found "particularly interesting". To go further, we run the test you suggest on the causal structure of our scenarios, which we vary by design. Each framework card states in advance which causal factors should change its verdict, so a fine-tuned model that follows its framework should shift its verdicts in the direction its card states when one of these factors changes.
We use the 150 held-out constructed scenarios, which were never seen in training. Each scenario was built with known values of three causal factors. For each framework and each factor, we compare how often the fine-tuned model approves the action under one value and under the other, and we do the same for its framework's references, which a model that follows its framework should match. We will add the full results in a new appendix:
-
Harm as a means rather than a side effect: Across all 16 frameworks, the fine-tunes change in the same direction as their references in 14. For example, Kantian fine-tunes, whose card prohibits using a person merely as a means, approve the action in 53% of the scenarios where the harm is a side effect but in only 20% where it is a means, close to the Kantian references (43% and 14%). Classical-utilitarian fine-tunes, whose card counts only outcomes, approve it about equally often either way (80% and 83%). In the other two, preference utilitarianism and classical pragmatism, whose cards do not treat this distinction as morally relevant, neither the references nor the fine-tunes change by more than about 3%. By contrast, a model that is only prompted with the Kantian description shows no such drop, so the effect comes from fine-tuning.
-
Harm that could have been avoided: The fine-tunes respond to this factor as their frameworks do. When the harm could have been avoided rather than happening anyway, the references of all 16 frameworks approve the action less often, and so do all 16 fine-tunes, by about the same amount (on average from 68% to 40% for the fine-tunes, and from 62% to 38% for the references).
-
Creating a benefit rather than preventing a harm: This makes little difference even to the references (46% against 52%). This matches Fränken et al. (2024), from whom our three factors are adapted, where a related factor (acting versus failing to prevent) had no clear effect on the judgments of humans, GPT-4 or Claude-2.
To summarize, on the causal factors that matter to the frameworks, the fine-tunes respond as their framework does.
We hope that these answers and the new results address the reviewer's concerns, and we kindly ask the reviewer to reconsider the score.
We thank the reviewer for the detailed review.
Presentation. In the revision, we will make the paper easier to follow. Section 4.2 will open by listing the four axes on which we evaluate:
We will also correct several counts, the typos and the citation format where applicable.
W1 and Q1. An independent validation needs several annotators with training in moral philosophy, enough to measure agreement between experts as well as with our verdicts. The differences between the analysis procedures set out in the framework cards are sometimes subtle, and some scenarios require considerable time for reflection (15 minutes or more), especially those that involve multiple parties and the calculation of a numerical approximation of the possible costs and benefits of the outcomes, so that it could not be done within the discussion period. The revision will state this limitation in Section 7. We can still address the concern in two ways: by measuring how sensitive our results are to a different reading of the cards, and with evidence that does not depend on our references.
On sensitivity, we can measure one source of disagreement now: a second generator's (DeepSeek-V4-Pro) reading of the same cards, which reaches the same verdict as ours on 77% of scenarios. Of the 23% of scenarios where DeepSeek-V4-Pro's verdict differs, a third are cases where one model answers "it depends" (conditional). Scored against that model's verdicts instead of ours, fine-tuned models still agree with their own framework far more than prompted ones: (\kappa = 0.404), against (0.258) for prompted instruction-tuned and (0.201) for prompted base models ((0.419), (0.286) and (0.227) against our references). The references are also based on expert input: a researcher with a doctorate in moral philosophy wrote each framework card and refined it over several rounds. Each card sets out the framework's key thinkers, its standard of evaluation, how it should respond to the causal structure of a dilemma, and its known limitations (Appendix B). MoReBench, in addition, does not use our references at all, and there each of the 36 fine-tunes scores highest on its own framework's rubric (W3 below).
For the concern that
"this limitation also affects the validity of the conclusions drawn about existing API models", we will phrase the API results as agreement with the card-based references. The references do not favour the model that wrote them: GLM-5.2, our generator, queried without a card, ranks 9th of 27 by mean (\kappa) against both its own references (a rerun) and DeepSeek-V4-Pro's, and DeepSeek-V4-Pro ranks 3rd against its own and 4th against GLM-5.2's.W2 and Q2. Fine-tuned models give a clear verdict far more often than prompted ones, and fine-tuning stays ahead in both recomputations you suggest. A clear verdict is an answer the judge can label yes, no or "it depends". How often prompted models give one depends strongly on the model: from 12 to 232 of the 250 scenarios (median per model, Table 6). Fine-tuned models give a clear verdict on 183 to 247 of the 250 scenarios, and half of them on 234 or more. The revision will add these numbers to Table 6. Fine-tuned models output an unclear verdict no more often than the reference verdicts they were trained on (7.6% of scenarios, against 8.7% for the references). Both recomputations compare each fine-tuned model with the same base model prompted with the framework, in its instruction-tuned and its base version, for 9 base models (18 comparisons), and a new appendix table will report them:
Same scenarios: On the scenarios where the fine-tuned model, the prompted model and the reference all give a clear verdict, the fine-tuned model agrees more with its framework in all 18 comparisons ((\kappa = 0.435) against (0.284) for instruction-tuned models, and (0.414) against (0.215) for base models).
Unclear verdicts counted: When an unclear verdict counts as a disagreement, the fine-tuned model agrees more with its framework for every framework in every comparison, and it also stays ahead in all 18 comparisons when "unclear" is treated as a verdict of its own.