Should, Say, Do: Evaluating Moral Consistency from Judgment to Tool Action in LLM Agents

As AI agents increasingly act on behalf of people and institutions, they will face situations where task objectives conflict with moral principles or where multiple moral considerations compete. Alignment can teach large language models (LLMs) to recognize and articulate moral principles, but knowing a principle does not ensure that it will guide behavior in such contexts. We introduce $\text{Should-Say-Do}$, a benchmark of 2,000 matched scenarios built by instantiating predefined ethical conflict structures across diverse deployment settings. Each scenario preserves the same situation and alternatives across three independently elicited conditions: a third-person judgment ($\text{Should}$), a stated choice in an assigned role ($\text{Say}$), and a tool action ($\text{Do}$). The benchmark covers moral temptations, where an attractive option conflicts with an identified principle, and ethical dilemmas with no designated correct choice. Across 14 LLMs, moral judgments and stated choices provide incomplete evidence about tool actions, with $\text{Should-Do}$ differences in reference agreement reaching 22.3 percentage points. For most models, the larger difference lies between $\text{Say}$ and $\text{Do}$, with substantial variation in magnitude across the model set. On ethical dilemmas, which carry no correct-answer label, choices also shift across conditions ($\text{Should-Do}$ absolute change of 19.3-47.2 points among models with repeated runs). These findings motivate evaluation and alignment methods that directly assess consistency across moral judgment, stated choice and tool action.