acceptodds
Under review as a conference paper at ICLR 2027

Pseudo-Deliberation: Measuring and Mitigating Value–Action Gaps in LLM Dialogue

Abstract

Large language models (LLMs) can articulate values that do not reliably translate into their behavior, a discrepancy known as the value–action gap. In this work, we argue that the value–action gap persists even under explicit reasoning, revealing a deeper failure mode we call “Pseudo-Deliberation”: the appearance of value-aware reasoning without corresponding behavioral alignment. We develop a framework for measuring and mitigating this failure across the value–reasoning–action pipeline. First, we introduce VALDI, which evaluates value–action alignment by tracing value commitments across self-surveys, explicit reasoning, and generated dialogue. VALDI instantiates this framework with 4,941 human-centered scenarios across five domains, three elicitation tasks, and five complementary measures of value adherence. Applying VALDI across instruction-tuned and reasoning-native LLMs reveals persistent value–action gaps and enables us to localize where value commitments are lost across the pipeline. Building on this decomposition, we introduce VIVALDI, a multi-agent intervention targeting both reasoning-stage value suppression and reasoning-to-action translation failures. VIVALDI decomposes deliberation into competing value-specific actions via debate and explicitly repairs commitments that would otherwise be lost before dialogue generation. Across models, this structured intervention improves value–action alignment by approximately 19–21% relative to standard chain-of-thought prompting. Together, VALDI and VIVALDI provide a framework for diagnosing and intervening on failures in translating articulated values into model behavior.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Artificial Intelligence, Human-AI Alignment, Multi-Agent Systems, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SkgjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper asks whether a model's stated values are reflected in its reasoning and final advice, and whether asking it to reason step by step improves this match. The authors introduce DAISY, a dataset of advice scenarios, and compare direct answers with four-step reasoning across eight models. A separate evaluator labels which Schwartz values appear in the responses. The authors find that reasoning does not improve value-action alignment and significantly lowers it for some models. They also find that values mentioned during reasoning often do not appear in the final advice, which they call "pseudo-deliberation." They then introduce ViVALDI, a multi-agent method intended to preserve these values, and report improvements on GPT-4o and Qwen.

Critical strengths
  • The prompts, data, and much of the analysis are available, so it was possible to understand how the study was run and check many of the results. I also appreciated the limitations section and the separate evaluator check.
  • Testing eight models is useful, and following values from an initial statement, through reasoning, and into final advice is an interesting setup.
Critical weaknesses
  • Fast and Slow differ in more than whether reasoning is shown. Fast tells the model to be decisive and avoid balanced or neutral answers. Slow does not include this instruction and adds a request to consider practical realities; for four models it also asks the model to reflect norms. In the GPT-4o data, Slow answers are also scored as neutral about twice as often as Fast answers. Because of this, I do not think the current comparison isolates the claim that reasoning itself lowers value-action alignment.
  • The model's initial form, the generation prompts, and the evaluator do not use exactly the same value definitions, and the evaluator is much more conservative about assigning values than the form. Some values also have special judging rules. Task 1 also asks about each value separately for one situation, while Schwartz's theory treats values as general goals whose relative importance and trade-offs matter. Some of what is described as value "loss" may therefore reflect measurement differences or a trade-off between competing values.
  • The ViVALDI result is also hard for me to interpret. Its repair prompt asks for additions whose removal would "materially weaken or change the recommendation," which is very close to the evaluator's own rule for deciding whether a value is present. The separate Claude evaluator uses the same basic rule. The headline improvement is also relative to CoT, which the paper itself finds is the weaker baseline. Compared with Fast, the gain is about 8%, and ViVALDI is tested on two models.
Questions for authors
  1. Could you run or report a Fast/Slow comparison where the instructions are matched and the only difference is the reasoning step? This is the result that would most affect my assessment.
  2. How much of the value-action gap remains if the same value definitions and more comparable thresholds are used for the initial form and the evaluator?
  3. Did you examine whether values that disappear are replaced by competing values in Schwartz's value structure? In the supplementary GPT-4o data, I found that when an encouraged value is missing from the advice, an opposing value is encouraged about twice as often as when the value survives (47% vs. 25%). This makes me wonder whether some "suppression" is actually a trade-off.
  4. Why is the empirical baseline of about 0.333 that appears in the analysis logs and Figure 5 not reported with the main results? How do the model scores compare with this baseline?
  5. Could you define "pseudo-deliberation" more precisely? At different points I understood it as reasoning lowering alignment, values appearing in reasoning but not advice, or value-aware reasoning not carrying into the answer. I am not sure these are the same claim.

I would add a short explanation of Schwartz's theory and define what the paper means by "values." Explaining the value structure and why the paper uses these 10 values would make the measurement choices easier to understand. "Other value orientations" is also mentioned as something the method could extend to, but I could not find an example of what this means.

I would also either support the claim that CoT should "naturally" improve value-action alignment with prior work, or just present it as a hypothesis. The discussion could do more to explain why the effects differ across values and models, and to consider explanations such as prompt wording, value trade-offs, and the evaluator itself.

8 comments
AuthorsOpenReview ↗

Author Response (1/2)

We thank the reviewer for finding the paper “interesting”, for recognizing that tracking values from initial statements through reasoning and final advice is an “interesting setup,” and for highlighting the usefulness of our evaluation across eight models. We also appreciate the reviewer’s positive assessment of our transparency with prompts, data, analyses, and limitations. We conduct 3 new experiments to address the reviewer’s concerns:

Instruction-matched Fast and Slow

We understand the concern associated between Fast and Slow’s prompts and how that may result in confounders which artificially inflate Fast’s performance. To address this, we conduct a prompt-matched analysis on 10% of our dataset using two different models (GPT-4o, Gemini only for cost), where Slow differs from Fast only in requiring an explicit reasoning format; all other prompt text remains identical.

Table 1: Comparison of prompt-matched Fast and Slow (N=500 scenarios). P-values are from two-sided paired permutation tests with 1,000 permutations.

ModelFast Alignment RateSlow Alignment RateΔ Rate (Slow − Fast)p-valueFast alignment distanceSlow alignment distance
GPT-4o0.45800.4435−0.01450.02800.17260.1731
Gemini0.47250.4770+0.00450.57140.19520.1923

As shown in Table 1, under prompt-matched conditions, Slow yields lower alignment than Fast for GPT-4o (Δ macro F1 = −0.0145, p = 0.0280). For Gemini, Slow yields a small improvement that is not statistically significant (Δ macro F1 = +0.0045, p = 0.5714). Matching the prompts changes the magnitude of the effect observed in our original experiments, but provides support for our central claim: explicit deliberation does not reliably improve value–action alignment.

Neutrality of Values

We thank the reviewer for raising this concern. We understand the concern that the original prompt setup produced more neutral labels for Slow, which could artificially reduce its alignment with Task 1. To address this, we conduct a neutrality analysis between our original setup and the new prompt-matched setup on 500 scenarios which we showed results for above in Tab 1. Table 2: GPT-4o’s neutrality analysis on N=500 scenarios.

MeasureOriginal FastOriginal SlowSlow/Fast RatioMatched FastMatched SlowSlow/Fast Ratio
Explicit neutral labels, all labels2.06%3.76%1.83×1.86%1.86%1.00×
Merged neutral and undetected (metric neutral)81.66%84.60%1.04×81.60%83.10%1.02×

As seen in Table 2, despite neutrality dropping in the prompt-matched setting compared to our old setting, we find the results from Table 1 to hold true, with Slow < Fast. We find that reduction in value-action alignment cannot be explained solely by the higher neutrality rate from the original Slow prompt.

Evaluator Definitions

We thank the reviewer for noticing this discrepancy. We would like to clarify our reasons for different evaluator labels.

  1. Different model families may be post-trained to acknowledge or ascertain different values. To ensure a clear target, we conducted various annotation studies between 2 humans and selected the best model and prompt that is substantially aligned to humans (obtaining kappa > 0.6).
  2. We believe this human validation is especially important for a human-centered task such as value identification. LLM-based judgments can vary substantially depending on the model post-training, prompt, etc, so downstream conclusions also depend on whether the evaluator reliably reflects human annotations. We therefore prioritize human–AI agreement (Appendix E) over using an evaluator solely because it shares the exact same definitions as Tasks 2/3.
  3. To further address the reviewer's concern, we demonstrate kappa between an evaluator that uses the same definitions as task 2/3 and that of our human-aligned evaluator in Table 3 below. For comparability, we run this on our original data, which we have human agreement and human-model agreement for.

Table 3: Kappa Differences Based on Evaluator Type

Kappa TypeHuman–HumanTask-Definition Evaluator–Human (Reviewer Suggestion)Human-Aligned Evaluator–Human
Detect0.810.590.67
Detect + Score0.810.560.62
AuthorsOpenReview ↗

Author Response (2/2)

Value Tradeoff Analysis

Our initial analysis consisted of analyzing different opposing values from Schwartz’s circular structure [1]. We test whether the disappearance of a value initially encouraged in reasoning was associated with encouragement of an opposing Schwartz value in the final advice. As seen in Table 4, some value pairs exhibit patterns consistent with a trade-off. For example, Security is more frequently encouraged when Self-Direction disappears (66.5% vs. 33.4%; r=0.327), but this is not consistent with raw patterns or correlations across all pairs. So value tradeoffs may explain some cases of suppression, but does not generalize for va-gap across all the Schwartz values in our data.

Table 4: Association Between Value Suppression and Opposing Value Encouragement

Suppressed ValueOpposing ValueOpposing value encouraged when suppressed (%)Opposing value encouraged when retained (%)Pearson r
Self-DirectionSecurity66.5%33.4%0.327
StimulationConformity1.8%0.0%0.064
HedonismTradition0.0%0.0%NA
AchievementBenevolence10.8%5.7%0.092
PowerUniversalism23.2%14.8%0.088
SecuritySelf-Direction56.4%25.5%0.280
ConformityStimulation2.2%0.0%0.103
TraditionHedonism6.7%0.0%0.194
BenevolenceAchievement15.9%2.6%0.245
UniversalismPower9.3%3.3%0.122

Empirical Baseline

Due to formulation of our 3-class structure, a naive, uniform random baseline would score 0.33 macro-F1. However, we see various levels of class imbalance within the Task 1 setup (Fig2a in manuscript). We therefore additionally report an empirical random baseline that samples predictions according to the Task 1 class distribution, providing a more realistic comparison under class imbalance.

Clarify Pseudo-deliberation

Prior work suggests that deliberation can improve value alignment [2,3], motivating our hypothesis that structured reasoning would reduce the value-action gap in AI's dialogue. Our results instead show that explicit reasoning does not reliably reduce this gap, and in several settings, can increase it. We originally used the term “pseudo-deliberation” to describe this pattern: models appear to engage with relevant values during reasoning, but those values are not reliably carried through to the final recommendation. We agree with the reviewer that the term may blur several distinct claims, particularly the distinction between reasoning lowering alignment and value-aware reasoning failing to translate into advice. To avoid this ambiguity, we will remove the term “pseudo-deliberation” and instead frame the paper directly around our empirical finding: the effect of reasoning on the value–action gap.

Clarifying Value Choices

  1. What are values and why Schwartz: We use “values” to refer to broad, trans-situational goals that guide individuals’ judgments and behavior. We agree this definition of what values are and how they guide decision-making is not explicit in our text. We will clarify this and state why we chose Schwartz, particularly due to its impact on recent work on AI and value alignment [4,5].
  2. Extendable to other value orientations: When we state that our framework is extendable, we refer to adapting definitions and evaluator criterion to frameworks such as Moral Foundations Theory and Rokeach Value Survey [6,7].

Supporting the Claim that CoT Improves VA-alignment

We found prior work supporting our initial hypothesis that structured reasoning/deliberation can improve value alignment [2]. We agree that this motivation should be more clearly foregrounded in the Related Work section. We will also expand our Discussion section to present different explanations for our heterogenous effects in alignment, including discussion of RLHF and differences in family-wise post-training, and evaluator/value/prompt choice.

[1] Schwartz, S. H. (2012). An overview of the Schwartz theory of basic values. Online Readings in Psychology and Culture, 2(1), 11.

[2] Nguyen-Hien, T. D., Teh, Y. W., Lee, W. S., & Zhi-Xuan, T. (2026). Rational Clarification by Assistive Agents via Value-of-Information Reasoning.

[3] Chakraborty, M., Wang, L., & Jurgens, D. (2025). Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework. EMNLP 2025.

[4] Shen, H., Clark, N., & Mitra, T. (2025). Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values? EMNLP 2025.

[5] Shen, H., Knearem, T., Ghosh, R., Yang, Y.-J., Mitra, T., & Huang, Y. (2024). ValueCompass: A Framework of Fundamental Values for Human-AI Alignment.

[6] Graham, J., Haidt, J., Koleva, S., Motyl, M., Iyer, R., Wojcik, S. P., & Ditto, P. H. (2013). Moral foundations theory: The pragmatic validity of moral pluralism. Advances in Experimental Social Psychology, 47, 55–130.

[7] Rokeach, M. (1973). The nature of human values. New York: Free Press.

Thank you for the detailed response and for running new experiments. The prompt-matched comparison is very helpful, and I think removing the term “pseudo-deliberation” makes the paper clearer.

With matched prompts, the drop for GPT-4o is much smaller than before (−0.0145 now, compared with −0.0378 in the original 500 scenarios). The p-value of 0.028 would not pass a Bonferroni correction for two models (threshold 0.025). The alignment distance is almost the same in both conditions (0.1726 and 0.1731), and Gemini shows no difference. So I read the new results as showing that reasoning does not help. They no longer show that reasoning makes things worse. Do you plan to update the abstract, the contributions, and the conclusion to match this? It would also help to see the matched prompt itself. If possible, please also run it on Llama or Qwen. These are smaller open models, and they were the ones with clear drops in the original Table 1.

Thank you for the trade-off analysis. Looking at your Table 4, in every pair the opposing value is encouraged more often when the original value is dropped. The only exception is Hedonism–Tradition, where both numbers are 0%. To me this looks like a consistent pattern that is strong for some pairs. Testing the broader Schwartz dimensions, and not only single pairs, might show this more clearly. Adding the number of cases in each row would also help, since some rows seem small.

About the evaluator. The values you report for the human-aligned evaluator (0.67 and 0.62) are the same as the Claude evaluator in the paper. The main results use the GPT ValueJudge, which the paper reports at about 0.62. Which evaluator is compared in your new table? Also, my question was about how much of the gap remains when the form and the evaluator use the same definitions and similar thresholds. Agreement with humans is useful, but alignment scores under that setup would answer the question directly.

Could you add the empirical baseline for each model to Table 1? This would show how far above chance each model really is.

Reference [2] uses “value” to mean value of information, which is about whether asking a question is worth it. It is not about human values, so I don’t think it supports the hypothesis. Reference [3] is relevant and would be a good addition to the related work.

Finally, my concern about ViVALDI was not addressed. The repair prompt uses almost the same wording as the evaluator’s rule for detecting a value. Also, the main gain is reported against chain of thought and not against Fast, which the paper itself shows is the stronger baseline.

Reviewer_T15tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies an interesting topic, the value-action gap and the pseudo‑deliberation failure mode in LLM Alignment. This work finds a previously overlooked issue: LLMs' reasoning can make LLMs talk about good values in their thinking but fail to act on them in the final dialogue, and CoT alone cannot fix the value‑action gap.

Thus, this work proposes VALDI equipped with the DAISY dataset to track value‑action gaps from self‑reported values through reasoning to final dialogue, and applies multi‑agent VIVALDI to revise reasoning and repair suppressed value commitments for higher output consistency.

Finally, this work benchmarks eight LLMs on the DAISY dataset and validates the multi‑agent VIVALDI intervention on GPT‑4o and Qwen3‑8B..

Critical strengths
  1. This work has identified the pseudo‑deliberation problem, a practical LLM alignment failure overlooked by prior work.

  2. The proposed VALDI can tracee the value propagation across the full pipeline: stated‑value, reasoning, and final dialogue.

  3. Experiments have proved that CoT cannot reliably fix the value‑action gap.

  4. Fine‑grained metrics are deisgined to capture value survival, suppression, and emergence dynamics.

Critical weaknesses
  1. This work relies heavily on the automatic ValueJudge, which may bring inherent subjective annotation risks.

  2. VIVALDI only optimizes internal consistency, not normatively correct human value standards.

  3. Evaluations are limited to advice‑giving dialogues, lacking other real‑world task scenarios. It is better to evaluate more.

  4. This work does not deeply explore how post‑training techniques trigger the pseudo‑deliberation phenomenon.

Questions for authors
  1. More discussion about the potential harmful risks brought by value‑consistent LLM outputs.

  2. Why VIVALDI‑R yields large performance gains on GPT‑4o but nearly no improvement on Qwen3‑8B?

4 comments
AuthorsOpenReview ↗

Author response (1/2)

We thank the reviewer for the constructive feedback and for recognizing the importance of the value–action gap, the usefulness of VALDI for tracing value propagation, and our fine-grained metrics. We address the reviewer’s concerns as follows:

Heavy Reliance on Automatic ValueJudge

We agree that our work relies heavily on an automatic evaluation procedure. To ensure that the LLM-based evaluator reflects human judgments, we first conducted multiple human annotation pilot studies, refining the annotation protocol until two human annotators reached substantial agreement (kappa = 0.81). We then evaluated the automatic judge against these human annotations, obtaining substantial human–AI agreement (kappa = 0.62). Thus, our automatic evaluator was not adopted without validation; it was selected and assessed against a human-validated annotation procedure developed through multiple pilots.

Internal Consistency, not Normative Alignment

We would like to clarify the aim of the paper: to demonstrate that there are “value-action gaps” between AI self-reports and AI dialogue. Hence, our primary goal is to increase internal consistency between what a model claims to value and how those values are reflected in its responses, rather than to determine normative human–AI alignment.

This distinction is particularly important in advice-giving settings, where models may explicitly endorse certain values while providing guidance that does not reflect them. VIVALDI is designed to reduce this internal value-action inconsistency, rather than determine what values are normatively correct. We will clarify this distinction more explicitly in the revised manuscript.

Evaluations are Limited to Advice-giving

We acknowledge this limitation, and even highlighted this shortcoming within our limitation section (Appendix A). However, we have reasons to believe that DAISY's setting is powerful compared to other datasets for evaluating VA-alignment:

  1. In advice giving and AI companionship, humans are vulnerable to sharing and have lowered judgement of the guidance they receive compared to other social reasoning domains [3-5]. When users are more likely to act on a model’s advice, discrepancies between the model's claimed values and actions have greater downstream consequences. We thus chose to focus on this under-explored, high-stake domain.

  2. In our tasks, AI is tasked to state values and be aligned with those values. Hence, the internal consistency and auditing of this is an “honesty” problem and is increasingly important with human reliance on AI.

  3. Lastly, this framework is still (1) extendable to other social reasoning tasks and datasets (norms, theory of mind) and (2) extendable to values other than Schwartz (Rokeach, Moral Foundations, etc [1,2]).

Post-training Techniques and Pseudodeliberation

We agree that our paper does not directly demonstrate how different post-training methods affect VA alignment. Prior work has begun to explore how post-training methods shape value tradeoffs and alignment, and we will better situate our work with respect to this literature [6]. Our goal is instead to assess VA alignment across model families and to investigate how these failures can be mitigated at inference time. This is important for proprietary models, where the exact post-training objectives and scale are not observable.

We will revise the Discussion section to make this scope clearer and to introduce concrete hypotheses for how different post-training objectives could produce the variation observed across models. We will explicitly connect these hypotheses to the differences reported in Table 1 and highlight causal analysis of post-training methods as an important direction for future work.

Vivaldi-R results between Qwen and GPT-4o

We note that VIVALDI-R generates a new dialogue from a revised reasoning trace produced through LLM self-debate. We hypothesize that the difference between Qwen3-8B and GPT-4o reflects model-family differences in how strongly revised reasoning influences downstream dialogue generation. Qwen3-8B may be less sensitive to changes in the reasoning trace alone, making it more susceptible to pseudo-deliberation when the intervention is restricted to the reasoning stage. In contrast, GPT-4o appears to propagate the revised reasoning more effectively into its final response. Both models perform better on the RD variant (paper's Table 2). We will clarify that the extent/power of Vivaldi R is largely dependent on the model family as well.

Discussion of Risks By Value-Aligned Outputs

We agree with the reviewer that this should be clarified. Despite yielding better VA alignment, VIVALDI may propagate values that the model initially considered but ultimately judged to be inappropriate or less relevant for the final response. Thus, stronger value consistency is not necessarily beneficial in every case and may introduce unintended effects. We will extend our Discussion section to dillineate this.

AuthorsOpenReview ↗

Author response (2/2)

[1] Graham, J., Haidt, J., Koleva, S., Motyl, M., Iyer, R., Wojcik, S. P., & Ditto, P. H. (2013). Moral foundations theory: The pragmatic validity of moral pluralism. Advances in Experimental Social Psychology, 47, 55–130.

[2] Rokeach, M. (1973). The nature of human values. New York: Free Press.

[3] Zhang, Y., Zhao, D., Hancock, J. T., Kraut, R., & Yang, D. (2026). Interaction with AI companions and psychological well-being. Nature Human Behaviour, 10, 1966–1980.

[4] De Freitas, J., Castelo, N., Uğuralp, A. K., & Oğuz-Uğuralp, Z. (2026). Mourning the loss of AI companions. Nature Human Behaviour.

[5] De Freitas, J., & Cohen, I. G. (2025). Unregulated emotional risks of AI wellness apps. Nature Machine Intelligence, 7, 813–815.

Dear reviewer,

Please let us know if you have any other questions or comments we can address. Thank you so much for your time and feedback!

Best, Authors

Reviewer_T15tOpenReview ↗

Keep my recommendation

This response did not provide enough new evidence or data to address my concerns. So I keep my recomendation unchanged.

Reviewer_UqmvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The authors are concerned about “pseudo-deliberation”, a variant of the action-value gap where agents explicitly reason about supporting different values without actually supporting them through their actions.
  • They construct DAISY, a dataset of value-laden situations where a user seeks guidance, by filtering, combining, and adapting existing value alignment and social intelligence datasets.
  • They then benchmark the value expression of different LLMs on DAISY scenarios across three settings: “fast” (baseline response), “slow” (baseline response after prompted CoT), and stated preferences, and design metrics to measure value-action gap.
  • They find that CoT actually slightly decreases alignment, and values that are frequently endorsed don’t uniformly exhibit high action alignment. In response to this, they present a multiagent scaffold which shows some improvement on their metrics.
Critical strengths
  • Misalignment between LLM stated values and actual actions is a major unresolved issue in current alignment evaluations; pseudo-deliberation is one way in which this could lead to negative downstream impacts from deployed agents.
  • The authors present a task that reasonably operationalizes this pseudo-deliberation effect, and conduct thorough quantitative analyses (e.g. error bars, different models, ablations) related to their findings.
Critical weaknesses
  • I think there are many potential explanations for the metrics differences that we’ve seen that are not the type of “pseudo-deliberation” that the paper is worried about, and it’s difficult to see whether this is actually the case without any established metrics or qualitative analysis. What’s actually happening when the custom metric identifies a value-action gap - is the model actually making commitments that it won’t honor, or is it just considering different options, tangentially mentioning values in a longer CoT, or paying lip service to values that the user brings up?
  • While VIVALDI modestly improves upon baselines, this is not necessarily surprising as it also has more compute than baselines. Without compute-matched baselines and more ablations on different components on the system, it’s unclear whether we actually need this prompting setup to be multiagent and/or iterative. Would you expect LLM inference to use this prompt by default, and if so, have you tested its impact on general alignment and capabilities to ensure it doesn’t degrade any other capabilities?
  • “Guidance-seeking dilemmas” seem to be an overly narrow environment, especially as value-action gaps in default chatbot settings seem to be less risky than value-action gaps in more agentic settings or sensitive domains. The main text doesn’t really justify why DAISY is needed over any of the existing datasets, or how DAISY is motivated by the problem description.
Questions for authors
  • The abstract highlights Vivaldi’s performance vs CoT, but isn’t this the worst-performing baseline? Why don’t we compare to the fast-thinking default or any of the baselines that you directly compare in Section 7?
  • What was the motivation for building the task/benchmark around Schwartz, and do you feel that all Schwartz values can cleanly map onto chatbot actions? This is a major methodological decision that isn’t really justified in the text.
4 comments
AuthorsOpenReview ↗

Author response (1/2)

We thank the reviewer for seeing value in our approach to a “major unresolved issue in current alignment evaluations” and appreciate their recognition of our robustness. We address the reviewer’s questions and concerns below.

Large Number of Explanations for “Pseudo-Deliberation”

We agree that our definition of pseudo-deliberation is intentionally broad, and that a quantitative approach could mask the true causal reason (lip service, tangentiality). Our evaluation is designed to measure the value–action discrepancy itself, as this discrepancy ultimately shapes what is communicated to the user. We provide qualitative examples in Table 1 to show distinct forms of VA misalignment.

Table 1: GPT-4o’s Sample Reasoning Trace and Final Dialogue, with Associated Evaluator Scores.

Reasoning evidenceReasoning stanceFinal dialogue evidenceDialogue stance
This decision would prioritize the long-term preservation of the ring over immediate adherence to tradition.Tradition: −1“This approach allows you to honor your family's legacy…”Tradition: +1
The potential benefits of ensuring accurate information and protecting the group's well-being may outweigh the discomfort of challenging the leader.Conformity: −1“Consider approaching the leader privately… you can help protect the group without immediately disrupting the dynamic.”Conformity: +1

In Table 1, we see two separate underlying mechanisms, with lip service for row 1 and social smoothing for row 2. From the user’s perspective, what matters is whether the advice faithfully reflects the model’s reasoning and can therefore be trusted, regardless of the mechanism causing the gap. We still agree with the reviewer’s concern, and will explicitly discuss these causal mechanisms as an important direction for future work.

Compute Matched Baseline

In Table 2 below, we present our analysis of various baselines on Qwen3-8B, from single agent baselines (Fast, Slow, State then reason) to multiagent baselines (GVIC). Rows are presented in increasing order of time complexity.

Table 2: Alignment Metric Scores for Qwen3-8B Ordered by Inference Complexity.

MethodInference complexityAlignment Rate ↑Distance ↓
FASTSingle-pass generation0.4420.179
SLOWSingle-agent CoT0.3980.190
State-Then-ReasonStated values + CoT0.4120.185
VIVALDI-R2 agents, 1 debate round + synthesis0.4430.178
VIVALDI-R&D2 agents, 1 debate round + reasoning audit/repair0.4800.164
GVIC5 agents, 3 update rounds + synthesis0.3760.203

Table 2 shows that Vivaldi RD surpasses all baselines, despite having modest inference complexity with only 1 round of iteration. We hypothesize this is due to the structure of intervention, rather than compute-sensitive advantage, as GVIC performs worse with significantly more compute usage.

Vivaldi Impact on General Alignment

We thank the reviewer for raising this concern. To address this, we perform a blind pairwise winrate analysis using the Harmlessness, Helpfulness, and Honesty paradigm [10] between VivaldiRD and the original Fast response, counting ties as half a point to each method. We sample the same 50 scenarios from GPT and Qwen and show results aggregated over 3 annotators in Table 3 below, with ties counting to half point per method.

Table 3: Blind Pairwise Win Rate (%) of Vivaldi-RD over Fast.

ModelOverallHelpfulHonestHarmless
Qwen3-8B68.89%75.00%66.67%65.00%
GPT-4o60.67%67.00%56.67%58.33%

Vivaldi-RD is preferred over Fast on all three HHH dimensions for both models, with overall win rates of 68.89% for Quen3-8B and 60.67% for GPT-4o. Our results suggest that Vivaldi-RD’s improvements in value-action alignment do not come at the cost of other alignment criteria.

Guidance-seeking dilemmas are narrow, and unclear why DAISY is needed

We agree that DAISY’s role should be further clarified. While DAISY focuses specifically on advice-giving, this domain is both underexplored and consequential for human reliance on AI. When users approach AI as an advisor, companion, or in a therapeutic-adjacent role, they may be especially vulnerable, with judgment already clouded by uncertainty or distress [7-9]. This makes the value-action gap particularly important: a model may claim to uphold certain values while ultimately providing advice that does not reflect them, and that advice may influence real-world actions. We will clarify this motivation and DAISY’s role in the manuscript.

AuthorsOpenReview ↗

Author response (2/2)

Comparing Vivaldi to CoT instead of Fast Thinking

We agree that comparing Vivaldi to CoT paints an incomplete picture, as GVIC < CoT in Sec 7. We chose CoT because ViVALDI is specifically a reasoning-based intervention, making standard step-by-step reasoning the most direct conceptual comparison to ViVALDI’s multi-agent guided reasoning.

However for a more fair comparison, we agree that comparison to Fast would be more reflective of overall outcome. We will revise the abstract to compare Qwen and GPT’s Vivaldi to their relative Fast thinking.

Schwartz Value Motivation

We address the two primary concerns below:

  1. Motivation for Schwartz: Compared to other value frameworks such as Moral Foundations Theory [1], Rokeach Survey [2], etc, we find Schwartz Theory of Universal Values to play the most foundation in human-AI alignment field’s recent works [3,4,5,6].

  2. One to one mapping of chatbot actions to values: We do not assume that every Schwartz value maps cleanly or uniquely onto every chatbot action. Multiple values may motivate the same action, and a single action may instantiate competing values. Our aim is to explore how these values guide a model’s reasoning and dialogue process, and if their representations can be corrected. We will clarify this in our manuscript.

[1] Graham, J., Haidt, J., Koleva, S., Motyl, M., Iyer, R., Wojcik, S. P., & Ditto, P. H. (2013). Moral foundations theory: The pragmatic validity of moral pluralism. Advances in Experimental Social Psychology, 47, 55–130.
[2] Rokeach, M. (1973). The nature of human values. New York: Free Press. GESIS Search [3] ValueCompass: A Framework of Fundamental Values for Human-AI Alignment

[4] Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?

[5] Assessing the Alignment of Large Language Models With Human Values for Mental Health Integration: Cross-Sectional Study Using Schwartz's Theory of Basic Values

[6] Exploring the Manifestation of Schwartz’s Basic Human Values in Large Language Models

[7] Zhang, Y., Zhao, D., Hancock, J. T., Kraut, R., & Yang, D. (2026). Interaction with AI companions and psychological well-being. Nature Human Behaviour, 10, 1966–1980.

[8] De Freitas, J., Castelo, N., Uğuralp, A. K., & Oğuz-Uğuralp, Z. (2026). Mourning the loss of AI companions. Nature Human Behaviour.

[9] De Freitas, J., & Cohen, I. G. (2025). Unregulated emotional risks of AI wellness apps. Nature Machine Intelligence, 7, 813–815.

[10] Bai, Y., Kadavath, S., Kundu, S., et al. (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv preprint arXiv:2212.08073.

Reviewer_UqmvOpenReview ↗

Response to Rebuttal

Thanks to the authors for their thorough response to my comments. I appreciate the commitment to revise the abstract and clarify the paper's grounding in Schwartz, as well as the human study showing that Vivaldi does not affect general alignment. Unfortunately, I still have significant reservations with the submission, which I clarify below, and am maintaining my score at this time.

Breadth of Pseudo-Deliberation Evaluations: I think the underlying issue here is that the framing of the paper very much presents pseudo-deliberation as a deep failure mode of LLMs that must be addressed. However, the way that the paper actually defines pseudo-deliberation is not reflective of the paper's discussion of this concept in the abstract and introduction. Without established metrics or a deeper qualitative analysis, I think this work's analysis of what pseudo-deliberation actually is falls short of the claims made in the paper. The fact that both qualitative examples presented seem to be lip service or social smoothing, rather than the model clearly making and failing to act on commitments, supports this point.

Compute-Matched Baselines: I appreciate the authors' rebuttal analysis of the compute used by different methods, but I don't actually think it addresses my concern that VIVALDI's improvement over existing methods is due to increased compute. Table 2 shows that VIVALDI still introduces additional inference complexity, and VIVALDI-R doesn't clearly outperform FAST even given the additional complexity. Comparing to compute-matched versions of FAST, SLOW, and State-Then-Reason could help identify whether the multi-agent and iterative components of VIVALDI are justified (it's not clear that we need multi-agent at all given that VIVALDI-R does not outperform FAST).

Role of DAISY: I agree that advice-giving is an important domain, but still feel that the DAISY contribution is overclaimed, given that it largely seems to be combining and rewriting existing datasets as advice-seeking scenarios.

AuthorsOpenReview ↗

Author response

Role of DAISY

We agree with the reviewer that DAISY should not be overclaimed as a completely new dataset of novel scenarios. Instead, its contribution is a unified advice-giving benchmark from existing resources, across various topics and subjects. We will soften language that overclaims DAISY as having novel scenarios, and will clarify that DAISY’s primary utility is to support systematic evaluation of the values that emerge in AI-generated guidance.

Breadth of Pseudo-Deliberation Evaluations

We wholly agree with the reviewer that our current analyses do not establish whether these inconsistencies reflect failed commitments, social smoothing, or another mechanism. To avoid this ambiguity, we have removed the term “pseudo-deliberation” and instead frame the paper directly around our empirical finding: the effect of reasoning on the value-action gap. We have revised the abstract, introduction, discussion so that the paper’s claims more closely match what our analyses directly establish.

VIVALDI-R and Compute-matched Baseline

We agree that VIVALDI-R does not have large gains compared to Fast thinking. We clarify a few points for the reviewer to clearly justify why VIVALDI-R is still useful despite not demonstrating large gains.

a. Value representation problem: Values may not be appropriately reasoned over. VIVALDI-R strives to produce value-rich deliberation, with dialogue being conditioned on this fix.

b. Reasoning to dialogue value transition problem: As we demonstrated in Slow thinking, explicit reasoning in general (modified or unmodified) is not sufficient to fix the VA gap. There must be some external mechanism that guides values to be consistent with reasoning. We actually believe that VIVALDI-R provides support for our aim that even reasoning that is further value-aligned does not yield better dialogue alignment. This motivates an explicit mechanism, such as VIVALDI-RD’s audit-and-repair stage, that enforces consistency between the reasoning and final dialogue.

c. Empirical justification for a and b above: To motivate understanding on why VIVALDI-R is still valuable, we provide judge’s scores within the Slow setting for reasoning and VIVALDI-R’s reasoning. Compared with Slow reasoning in Tab 1, VIVALDI-R substantially improves reasoning alignment from 35.40% to 53.17% (~17 percentage points). This suggests that the multi-agent intervention is effective at improving explicit value deliberation, supporting (a). At the same time, this improvement is not fully reflected in the final dialogue, where VIVALDI-R reaches 47.00% alignment. This gap supports (b): improving value representation during reasoning alone is insufficient, motivating an explicit mechanism to preserve that alignment when translating reasoning into the final response.

Table 1: GPT-4o’s Alignment Improvement Breakdown in Vivaldi. In Reasoning text type, we leverage valuejudge to assess alignment of reasoning traces to model’s Task 1 vectors. In Dialogue text type, we assess dialogue alignment to Task 1 vectors. As seen, both Vivaldi variants boost alignment at different stages, with RD ultimately demonstrating best performance.

Text TypeMethodAlignment Rate
ReasoningSlow35.40%
VIVALDI-R (post-synthesis)53.17%
DialogueFast45.32%
Slow41.54%
VIVALDI-R47.00%
VIVALDI-RD49.14%

d. Compute matched baseline: We agree with the reviewer that a compute-matched baseline would help distinguish the contribution of VIVALDI’s structured multi-agent intervention from the contribution of additional inference compute. Because running the full evaluation is computationally expensive and was not feasible time-wise within the rebuttal window, we have not added this experiment here. In our revision, we will add a single-agent self-refinement baseline that uses the same number of sequential inference stages and approximately the same token budget as VIVALDI RD.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.