acceptodds
Under review as a conference paper at ICLR 2027

ValueGraph: Evaluating and Representing Value Trade-offs in Large Language Models

Abstract

Language models that advise on decisions often let one value give way to another. Value evaluations mostly summarize this as orientations toward individual values, leaving open how specific conflicts are resolved and in which contexts. We introduce ValueGraph, a graph of value trade-offs in which nodes carry value orientations and directed edges record pairwise priorities with their direction, magnitude, robustness, and supporting scenarios. We construct 2,791 scenarios over Schwartz's 19 refined values, covering all 171 value pairs across eight human flourishing domains. Human validators endorse the scenarios' realism and conflict structure and agree with the LLM judges that accept them. Adapting best-worst scaling, we ask models which of three actions deserves most and least consideration. Applied to 22 models, ValueGraph reveals three main findings: First, models with highly similar value rankings can still diverge sharply on specific trade-offs. Second, all models share a boundary in which six values, mostly Benevolence and Universalism, precede five others such as Power-Dominance and Hedonism, yet no robust order holds within the favored group. Third, robust priorities reverse across domains, often in the same direction across models. These findings inform that alignment targets should specify which value takes precedence and in which contexts.

Primary area:
datasets and benchmarks
Keywords:
Large Language Models, Value Alignment, Value Trade-offs, Schwartz's Values, LLM Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_L4J9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ValueGraph, a graph-based approach for studying how language models make trade-offs between competing values. By building 2,791 scenarios covering Schwartz’s 19 refined values and eight decision domains, the authors represent models’ value orientations and pairwise priorities in a unified graph structure. Experiments on 22 language models show that models with similar overall value rankings may still differ substantially in specific trade-offs, and that some value priorities change depending on the decision context. Through robustness checks and a prompting study, the paper further demonstrates that ValueGraph can help analyze value alignment beyond simple value rankings.

Critical strengths

1. The paper identifies a meaningful limitation of global value rankings. In Section 5.1, Qwen 3.8 Max and Claude Opus 5 achieve a highly similar overall ranking (ρ = .960), yet show differences of roughly 35–37 percentage points on specific trade-offs involving Self-direction–Thought versus Face and Achievement. This demonstrates that similar global value profiles can still correspond to different decision behaviors, motivating the need for evaluations that preserve local value conflicts rather than collapsing them into a single ordering.

2. The dataset design enables systematic analysis of both shared patterns and contextual variation. By covering all 171 value pairs and including different third-value combinations and domains, the dataset supports robustness analysis beyond isolated conflicts. The shared six-by-five value boundary identified in Section 5.2 provides a more precise characterization of common model tendencies, while the domain-level analyses reveal where priorities change across contexts.

Critical weaknesses

1. The incremental contribution over related work is not sufficiently clear. Prior work, including ConflictScope (Liu et al., 2025) and Priority Graph (Tang et al., 2026), has explored value conflicts, contextual priorities, and graph-based representations. ValueGraph extends these directions with broader value-pair coverage and robustness analysis. However, the paper does not clearly establish what new insights are enabled by the graph representation itself beyond organizing pairwise preferences and contextual information. The findings on hidden local trade-offs and domain-dependent priorities are valuable, but the contribution would be stronger if the paper demonstrated cases where ValueGraph enables analyses or interventions that simpler pairwise evaluations would miss.

2. The validation does not fully establish whether the reported trade-offs are robust across elicitation protocols. Section 4.2 mainly validates BWS through correlations of overall value rankings. However, the main claims concern pairwise priorities and domain-specific trade-offs, which may not be captured by ranking-level agreement alone. Since BWS measures consideration while protocols such as P3 measure direct choices, evaluating pairwise relations across protocols would provide stronger evidence that the reported trade-offs reflect stable model behavior rather than properties of the BWS setup.

Questions for authors

1. What does ValueGraph add beyond ConflictScope and Priority Graph? In particular, which of the paper’s findings rely on the graph representation or robustness analysis, rather than on the underlying pairwise preferences and domain-level analyses?

2. For the models evaluated with both BWS and P3, how consistent are the resulting pairwise priorities across protocols? Do the shared value boundaries and domain reversals observed with BWS also hold under P3, or are they sensitive to the elicitation method?

3. In Section 6, which parts of the context-rule prompt are derived from ValueGraph analysis, and which are specified by the target policy? Does the experiment demonstrate that ValueGraph helps identify alignment targets, or mainly that models can follow an explicitly provided rule?

7 comments
AuthorsOpenReview ↗

Responses to Reviewer L4J9 (1/4)

Thank you for highlighting the strengths of our finding on what global value rankings hide and of our systematic dataset design. Below, we respond to each weakness and question.

1. (W1, Q1) Contribution over Related Work

W1-1. "The incremental contribution over related work is not sufficiently clear."

Q1-1. "What does ValueGraph add beyond ConflictScope and Priority Graph?"

The key distinction is that ConflictScope primarily elicits and aggregates value preferences, Priority Graph provides a conceptual representation of priorities, whereas ValueGraph develops a quantitative, robustness-aware framework for analyzing the structure of individual value trade-offs. The differences are reflected in both the methodology and the resulting findings.

Table 1. How each work treats the priority between two values.

WorkPairwise priorities areRobustness is tested forOutput
ConflictScope [1]aggregated into a Bradley–Terry ranking of the values in each value setthe ranking, across domains and prompt templatesa model's overall value ordering, and its shift between multiple-choice and open-ended evaluation
Priority Graph [2]formalized conceptually, with the measurement function left generalnothing (no measurement)arguments on dynamic priorities and priority hacking
ValueGraphestimated one by one: an empirical edge for each of the 171 pairs, kept separate from the BT scoreseach edge, as a margin over 26 material checks (domain and third-value deletions)which individual trade-offs are stable, contested or context-sensitive

(a) ConflictScope: Binary Conflicts and Aggregate Rankings vs. Systematically Characterized Pairwise Relations.

ConflictScope [1] generates two-value conflicts and primarily investigates how model preferences differ between multiple-choice and simulated-user open-ended evaluations, summarizing these preferences through Bradley–Terry rankings [3]. Its domain and prompt-template analyses assess the consistency of preferences across evaluation conditions.

ValueGraph advances this approach in three ways:

  • Systematic multi-value coverage: Rather than evaluating isolated binary conflicts, we enumerate all 969 triples of Schwartz's 19 refined values [4], covering all 171 value pairs. Each pair is observed alongside different third values, enabling analysis of whether its priority depends on other competing values.
  • Independent empirical edges: We retain pairwise priority estimates separately from global BT orientations. Each edge records its direction, magnitude, and supporting scenarios, allowing direct examination of relations that aggregate rankings obscure.
  • Edge-specific material robustness: We introduce a robustness margin computed over 26 systematic material conditions, including domain and third-value deletions. Unlike aggregate domain-agreement statistics, this identifies which individual priority relations survive changes in evaluation materials and which are sensitive to particular contexts.

These differences change the analytical output: rather than only characterizing a model's overall value ranking or its stability across evaluation settings, ValueGraph identifies the precise relations that are stable, contested, or context-sensitive.

(b) Priority Graph: Conceptual Formalization vs. Measurable and Testable Priority Structures.

Priority Graph [2] formalizes context-dependent priority relations among instructions and values, but leaves the measurement function general and primarily develops conceptual arguments concerning dynamic priorities and priority hacking.

ValueGraph makes these relations empirically measurable. We define explicit estimators for pairwise priority strengths, material robustness margins, and graph-level distances; link each relation to its underlying scenarios and responses; and instantiate the framework across 2,791 scenarios and 22 models.

This transforms a conceptual priority representation into a quantitative auditing framework capable of testing cross-model agreement, identifying robust relational structures, and locating context-dependent changes.

AuthorsOpenReview ↗

Responses to Reviewer L4J9 (2/4)

1. (W1, Q1) Contribution over Related Work (continued)

W1-2. "... the contribution would be stronger if the paper demonstrated cases where ValueGraph enables analyses or interventions that simpler pairwise evaluations would miss."

An evaluation that reports one win rate per pair cannot tell which differences, shared priorities and reversals survive a change of materials. The domain-conditioned edges also support an intervention: they locate the two domains where the alignment target of §6 has to differ. There, in both models of §6, a context-free prompt raises agreement with the target when the two domains are pooled (by 16.2 and 9.8 points), yet lowers it in Health (by 22.2 and 7.5 points; see Q3).

(c) What new findings are enabled by these components?

Our three principal findings illustrate their distinct contributions.

Hidden local differences (§5.1). Qwen 3.8 Max and Claude Opus 5 exhibit highly similar global value rankings (Spearman ρ = .960), yet differ by 37.3 and 35.3 percentage points on specific trade-offs involving Self-direction–Thought. These differences become visible when empirical pairwise priorities are retained rather than summarized solely through global rankings.

Robust shared structure without a universal total ordering (§5.2). Across 22 models, we identify 61 relations with consistently retained directions, including a complete 30-edge bipartite subgraph in which six favored values precede five others. None of the 15 relations within the favored group is robustly retained across all models. This reveals a shared priority boundary alongside unresolved internal trade-offs: information that a single global value ranking cannot adequately characterize.

Persistent domain-associated reversals (§5.3). By examining conditional edge priorities and their supporting materials, we identify 242 persistent reversals, twice the 121 found when domain labels are permuted. This allows us to locate which specific value relations change direction and in which decision domains, rather than reporting only aggregate contextual variation.

Q1-2. "... which of the paper's findings rely on the graph representation or robustness analysis, rather than on the underlying pairwise preferences and domain-level analyses?"

All three findings rely on the robustness analysis, and §5.1 and §5.2 also on the graph representation. §5.1 compares two graphs at both levels. Their node rankings agree, while the robustness margin shows that Claude Opus 5 retains Self-direction–Thought over Face and over Achievement and Qwen 3.8 Max retains neither direction. An edge is retained when its favored value wins at least 60% of comparisons in each of the 26 material checks. The 985 between-model gaps that §5.1 reports as significant after family-wise correction all keep their sign under each of these checks. In §5.2, the boundary consists of edges retained in every one of the 22 models, and its form is a property of the graph: a complete six-by-five bipartite subgraph whose favored side has no edge retained in every model. In §5.3, a reversal is counted only if it persists when any single scenario is deleted.

In summary, ValueGraph's contribution is the joint estimation of global orientations, empirical priority relations, and edge-specific robustness, enabling structural and context-sensitive analyses that neither global ranking summaries nor conceptual priority graphs provide. We will make these distinctions explicit in the main text and add a direct methodological comparison with ConflictScope and Priority Graph.

AuthorsOpenReview ↗

Responses to Reviewer L4J9 (3/4)

2. (W2, Q2) Consistency of Pairwise Priorities across Protocols

W2. "Section 4.2 mainly validates BWS through correlations of overall value rankings. ... evaluating pairwise relations across protocols would provide stronger evidence that the reported trade-offs reflect stable model behavior rather than properties of the BWS setup."

Q2. "... how consistent are the resulting pairwise priorities across protocols? Do the shared value boundaries and domain reversals observed with BWS also hold under P3, ..."

Thank you for the suggested analysis. The reported trade-offs are not properties of the BWS setup. As a control, we presented the same ValueGraph scenarios as direct choices between two actions (P3 in §4.2) and rebuilt the graph from these choices alone: the retained priorities, the shared boundary and the domain effects all reappear (Table 2). Both graphs use the same scenarios, the paper's estimator and its robustness margin, on the four models of §4.2. They differ only in what is asked: BWS asks which of three actions deserves most and least consideration, the P3 control which of two actions to choose. An edge is retained when its favored value wins at least 60% of comparisons in each of the paper's 26 material checks.

Table 2. Graphs built from BWS and from the P3 control on the same scenarios.

Claude Sonnet 5DeepSeek V4 ProGemini 3.7 FlashGPT-5.6 Sol
Correlation of edge strengths, BWS vs. P3 control (r, 171 pairs).975.894.980.968
Edges retained under BWS that keep their direction under the P3 control126/126105/107121/121115/116
Shared-boundary edges (§5.2) with the same direction under the P3 control30/3030/3030/3030/30

Of the 470 edges retained under BWS, 467 keep their direction under the P3 control and none is retained in the opposite direction, and all 30 boundary edges keep their direction in all four models. In these four models, holding the triple fixed, the priority differs across domains (F test, p < .05) for 27 of the 171 pairs under BWS and 35 under the P3 control, against 8.6 expected by chance. The two sets share 17 pairs, against 5.5 if they were unrelated. The domain shifts agree between the two formats (r = .82 over pair-by-domain cells). A cell's shift is the pair's win rate in that domain minus the average of the same triples across domains. The Section 6 contrast is also recovered. Leaving out one triple at a time, Health and Personal Development are selected as the domains where Conservation gains and loses most in all 600 fits under the P3 control. They differ by 22.0 points on the left-out triples.

We also ran the reverse control, keeping the BWS question and replacing the scenarios. Three models other than the paper's generator (DeepSeek V4 Pro, Gemini 3.1 Pro and GPT-5.6 Sol) wrote 1,791 new scenarios for the same value triples and domains, and eight of the evaluated models answered them, the four of Table 2 among them (Table 3). Of the 962 edges the paper retains for these eight models, 926 keep their direction, 20 reverse and 16 are tied, and all 30 boundary edges keep theirs in each of the eight models.

Table 3. The same edges on the 1,791 new scenarios, for the four models of Table 2.

Claude Sonnet 5DeepSeek V4 ProGemini 3.7 FlashGPT-5.6 Sol
Edges retained in the paper that keep their direction on the new scenarios124/126101/107112/121112/116
Shared-boundary edges (§5.2) with the same direction on the new scenarios30/3030/3030/3030/30

BWS remains the main protocol for two reasons. It follows the best–worst scaling design of human value research [5, 6], and one BWS answer yields the three pairwise decisions that the P3 control collects with three separate questions, so each scenario needs one question instead of three. We will add this comparison of BWS and the P3 control to the appendix of the revised version.

Reviewer_12s3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ValueGraph, a framework for evaluating value trade-offs in LLMs beyond overall value scores or rankings. The authors construct 2791 decision scenarios covering 19 Schwartz values and eight domains, and evaluate 22 models using a best-worst scaling protocol. ValueGraph represents overall value orientations as nodes and pairwise priorities as edges, including their strength, robustness, and supporting scenarios. The main findings are that similarly ranked models can differ in specific trade-offs, models share some broad priorities but not a consistent ordering within favored values, and priorities can reverse across domains. The authors validate the dataset through LLM judges and human evaluation, compare different elicitation protocols, and conduct robustness analyses. A final prompting experiment illustrates the usefulness of context-specific value evaluation.

Critical strengths

1. Motivation & Usefulness

  • Value alignment in LLMs is a timely topic, and I agree that evaluations based on individual scores or rankings may overlook conflicts between values. Although studying value conflicts is not new, I think this is an important direction.
  • The proposed dataset seems useful, covering all 171 value pairs across eight domains. The finding that similarly ranked models can differ in specific trade-offs is also interesting (§5.1).

2. Related Works & Novelty

  • While graph-based representations have been explored before, I find ValueGraph a useful extension, combining value orientations, pairwise priorities, robustness, and supporting scenarios.
  • The systematic coverage of value pairs across models and domains also adds value.

3. Experiments & Robustness

  • The dataset construction seems careful, with two independent LLM judges and human validation (§2.2).
  • The experiments cover 22 models, with five alternative elicitation protocols evaluated on four models (§2, Table 1). The statistical analyses also help support the main findings.

4. Presentation

  • The paper is generally well organized and easy to follow. The figures and concrete examples help explain the method and findings.
Critical weaknesses

Overall, I lean toward weak acceptance, as I find the dataset and empirical findings useful. However, I am not fully convinced that ValueGraph provides much beyond simpler scalar profiles, and some aspects of the evaluation need stronger validation. I think these concerns are important but potentially addressable.

1. Related Works & Novelty

  • The paper lacks a dedicated related work section in the main text, with most discussion in App. B.2. Several relevant works seem to be missing. For example, value-rating instability and ranking-based evaluation are also studied in VALUEFLOW [1], while Utility Engineering [2] studies pairwise preferences and utility representations. Other works [3,4] are also relevant to context-dependent preferences and evaluation reliability, although [3] is a recent preprint and a full comparison is not expected.
    • I think more direct comparisons would help clarify what ValueGraph adds beyond previous approaches.

2. Experiments & Robustness

  • My main concern is whether ValueGraph captures meaningful information beyond scalar profiles. In Figure 6 (§5.1), Qwen and Claude have similar rankings (ρ = .960), but their BT scores also differ, which may already explain part of the pairwise differences. Also, the 73.9% reconstruction accuracy (§4.2, L311-312) is in-sample.
    • A direct comparison with BT/PL-predicted probabilities on held-out scenarios would be useful. The different weighting of node and edge estimates (App. F.1, L1404–1407) should also be controlled.
  • The interpretation of BWS responses as pairwise value priorities needs stronger validation. Each three-action ranking yields three comparisons (§2.1, L139–141), but the relative preference between two actions may depend on the third, which the third-value deletion in §3.3 does not directly test. Also, consideration is not necessarily the same as choice.
    • Section 4.2 (L309–312) compares node rankings across protocols, not edges. Since P3 directly measures pairwise choices, an edge-level comparison would be useful.
  • I also have some concerns about dataset construction and human validation. Claude Sonnet 5 generated the scenarios, GPT-5.6 Sol served as a judge, and both are among the evaluated models (§2.2, L156–159). Although the independent judging and human validation help, this overlap could still affect the scenario framing and results.
    • Also, human validators provided most/least choices (App. E.4, L1224–1227), but these are not directly compared with model choices. Given the shared boundary finding (§5.2), I think a matched human–model comparison would provide useful additional validation.

References

[1] Kim et al., VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models, ICML 2026.

[2] Mazeika et al., Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs, NeurIPS 2025.

[3] Trhlik et al., LLMs Contain Multitudes: How Deployment Context Reshapes Model-Level Preferences and Values, arXiv 2026.

[4] Shen et al., Revisiting LLM Value Probing Strategies: Are They Robust and Expressive?, EMNLP 2025.

Questions for authors
  1. Scalar baselines (§5.1, Fig. 6): How much of the observed pairwise differences is already explained by BT or Plackett–Luce scores? Could the authors compare held-out pairwise predictions with matched weighting, including domain-conditioned scalar models?
  2. Edge-level validation (§4.2): Do BWS and P3 produce similar pairwise priorities, rather than only similar node rankings? Do the main findings remain consistent across these protocols?
  3. Dataset and human validation (§2.2, App. E.4): Could the authors compare the recorded human BWS choices with model responses on the same scenarios? Also, how might the overlap between dataset-construction models and evaluated models affect the findings?
10 comments
AuthorsOpenReview ↗

Responses to Reviewer 12s3 (1/4)

Thank you for highlighting the strengths of our focus on value conflicts, full value-pair coverage, and combined graph representation. Below, we respond to each weakness and question.

1. (W1) Comparison with Related Work

W1. "The paper lacks a dedicated related work section in the main text ... more direct comparisons would help clarify what ValueGraph adds beyond previous approaches."

The paper reviews related work in Appendix B.2. In the revised version, we will add the comparison with the suggested references to clarify what ValueGraph adds, and move the related work review into the main text. We have examined the suggested references closely and identify four substantive differences in what is measured, how trade-offs are elicited, and what conclusions can be drawn.

Table 1. What each work measures and reports.

WorkMeasuresReportsPriority between two values in conflict
VALUEFLOW [1]how strongly a text expresses one valuean intensity scale per valuenot measured
Utility Engineering [2]preferences over world-state outcomesa utility per outcomenot measured; outcomes are not mapped to values
LLMs Contain Multitudes [3]preferences over outcomes under four task framingsa utility per outcome and framingnot measured
Revisiting LLM Value Probing [4]survey-based value profilesa profile per model, correlated with action ratingsnot measured
ValueGraphmost and least considered of three actions, each motivated by a different valueBT scores and one edge per value pair, with robustness and supporting scenariosall 171 pairs of 19 values, in eight domains

(a) VALUEFLOW: Value Intensity vs. Value Priority.

VALUEFLOW [1] measures how strongly a text expresses a given value, using pairwise text rankings and Plackett–Luce aggregation. Its intensity estimates are constructed separately for each value. This answers "How strongly is value A expressed?" but does not directly answer "When A and B compete, which takes precedence?"

ValueGraph directly addresses the latter. By constructing scenarios where different values motivate competing actions under explicit constraints, we estimate the relative priority between values, rather than their individual expression intensities. High endorsement of two values does not determine how a model resolves a conflict between them.

(b) Utility Engineering: Outcome Utilities vs. Interpretable Human-Value Trade-offs.

Utility Engineering [2] elicits pairwise preferences over possible world-state outcomes and fits latent utility models to study coherence, consistency, and control.

ValueGraph instead maps actions to Schwartz's 19 refined values [5] and systematically measures all 171 value pairs. More importantly, our empirical edges preserve pair-specific priorities independently of the global Bradley–Terry scores [6], together with their supporting scenarios and robustness across material variations. This allows us to identify which specific human values take precedence, whether that relationship is stable, and where aggregate value orientations conceal local differences.

(c) LLMs Contain Multitudes: Deployment Framing vs. Situated Value Conflicts.

Trhlik et al. [3] examine how preferences over fixed outcomes change when the surrounding deployment task changes (e.g., writing a news article versus a Reddit post).

ValueGraph investigates a different dimension: how the relative priority of two explicitly defined human values varies across substantive decision domains. We estimate domain-conditioned priority relations across eight human flourishing domains and systematically examine their robustness and reversals. Unlike outcome-level contextual preference shifts, these results identify the particular value conflicts involved and the contexts in which their priorities change.

(d) Revisiting LLM Value Probing: Value Profiles vs. Explicit Trade-off Decisions.

Shen et al. [4] compare survey-based value probing strategies and assess whether the resulting value representations correlate with action preferences.

ValueGraph measures trade-offs directly within a shared decision: three actions, each representing a distinct value, compete under the same situational constraints. The resulting pairwise relations reveal how values are prioritized relative to one another, rather than whether independently measured value scores correlate with action ratings.

Overall, ValueGraph moves beyond individual value intensities, outcome utilities, and survey-based orientations to measure the structure of situated human-value trade-offs. Its combination of complete value-pair coverage, empirical priority edges, and material robustness enables systematic identification of local model disagreements, shared priority boundaries, and domain-associated reversals across 22 models: analyses not provided by these prior frameworks.

AuthorsOpenReview ↗

Responses to Reviewer 12s3 (2/4)

2. (W2, Q1) Scalar Baselines

W2-1. "My main concern is whether ValueGraph captures meaningful information beyond scalar profiles. ... the 73.9% reconstruction accuracy ... is in-sample."

Q1. "How much of the observed pairwise differences is already explained by BT or Plackett–Luce scores? Could the authors compare held-out pairwise predictions with matched weighting, including domain-conditioned scalar models?"

Thank you for the suggested comparison. Beyond the scalar profile, ValueGraph records two things for each priority, whether it is robust and whether it changes with domain, and both are borne out on scenarios the scores were not fitted on. The BT scores are the graph's nodes and give each priority's average direction. In Figure 6, they imply smaller gaps between the two models than observed (24 and 27 points against 37.3 and 35.3) and put both on the same side of Self-direction–Thought versus Face: they give Self-direction–Thought a 64% probability in Qwen 3.8 Max, yet it wins only 46% of Qwen's comparisons. The graph shows the difference the scores miss: both Figure 6 edges, over Face and over Achievement, are robust for Claude Opus 5, and neither is for Qwen 3.8 Max. In place of the in-sample 73.9%, we test robustness on new scenarios written by other generators and domain on held-out scenarios. Plackett–Luce scores match BT (73.91% against 73.93% in-sample; App. F.3), so the comparisons use BT.

Robustness (Table 2). A score gap gives the average direction of a priority, not whether that priority is robust. The graph calls an edge robust, and retains it, only if the favored value wins at least 60% of comparisons in each of the paper's 26 material checks. Across the 22 models, the scores give the favored value a winning probability of at least 60% on 3,013 of the 3,762 model-edges (171 pairs in each model), and the graph finds 555 of them (18.4%) not robust. We checked the BT-predicted priorities on scenarios newly written by other generators, which neither the scores nor the graph were fitted on (eight models): 1.3% of the robust priorities reverse (the other value wins the majority of comparisons), against 20.0% of the others, and the gap holds at the same BT probability: 5.9% against 24.5% at 60–70% (Table 2; 95% CI of the difference 4.4 to 32.7). The graph thus tells which score-implied priorities can be relied on.

Table 2. BT-favored priorities that reverse, with the other value winning most comparisons, on other generators' scenarios (eight models; ties excluded).

BT probability that the favored value winsGraph: robust (edge retained)Graph: not robust (edge not retained)
60–70%5.9% (7 of 119)24.5% (34 of 139)
≥ 70%0.6% (5 of 796)8.9% (5 of 56)
All1.3% (12 of 915)20.0% (39 of 195)

Domain (Table 3). A global score gives each value one strength in every domain; the graph's domain-conditioned edges test whether a priority changes with domain. In the 22 models evaluated on our full dataset, 30 of the 171 pairs differ across domains when the triple is held fixed (F test against the remaining variation between scenarios, p < .05; 8.6 expected by chance). For the held-out comparison with matched weighting, we predicted the prioritized value in held-out scenarios (five folds of whole triples), fitting every predictor on the same comparisons with equal weights, which removes the weighting difference of App. F.1. The global scores, domain-conditioned scores (one score per value in each domain) and domain-conditioned scores with the graph's edge terms added (per pair and per pair in each domain) pick the same winner for more than nine in ten choices (overall accuracy 73.7%, 74.8% and 75.0%), so their accuracy can differ only where they disagree, and we compare them there: the predictors with domain conditions are right 57% of the time and the global scores 43%, in each of the 22 models (Table 3). The domain-conditioned scalar model gains for the same reason as the graph: priorities change with domain, which a single profile cannot express.

Table 3. Held-out choices on which a predictor with domain conditions and the global BT scores pick different winners (22 models; all fitted with equal weights).

PredictorHeld-out choices where it disagrees with the global scoresIt is rightGlobal scores are rightModels where it is right more often
Domain-conditioned scores7.7%56.9%43.1%22 of 22
Domain-conditioned scores + graph edge terms (pair, pair × domain)8.8%57.2%42.8%22 of 22

The robustness of each priority is what no set of scores, global or domain-conditioned, records. We will add the robustness comparison and report domain-conditioned scores as a compact summary of the graph's domain conditions in the revised version.

AuthorsOpenReview ↗

Responses to Reviewer 12s3 (3/4)

3. (W2, Q2) Edge-level Validation

W2-2. "The interpretation of BWS responses as pairwise value priorities needs stronger validation. ... the relative preference between two actions may depend on the third ... consideration is not necessarily the same as choice."

Q2. "Do BWS and P3 produce similar pairwise priorities, rather than only similar node rankings? Do the main findings remain consistent across these protocols?"

BWS and P3 give similar pairwise priorities (Table 4): of the 470 edges retained under BWS, 467 keep their direction under P3 and none is retained in the opposite direction. Reading best–worst responses as pairwise priorities follows human value research [7, 8]. P3 presents the same scenarios as three separate two-action choices, so we compared edges, applying the paper's estimator and robustness margin to each protocol, on the four models of §4.2. An edge is retained when its favored value wins at least 60% of comparisons in each of the paper's 26 material checks.

Table 4. BWS versus P3 at the edge level, on the same scenarios.

Claude Sonnet 5DeepSeek V4 ProGemini 3.7 FlashGPT-5.6 Sol
Correlation of edge strengths (r, 171 pairs).975.894.980.968
Edges retained under BWS that keep their direction under P3126/126105/107121/121115/116
Shared-boundary edges (§5.2) with the same direction under P330/3030/3030/3030/30
  • Dependence on the third action: Human choice research finds that a third option can change the preference between two others [9, 10]; here, the third action does not decide a priority's direction. P3 shows only two actions at a time, and the retained edges keep their direction there. Within BWS, in all 22 models, we compared the same pair across triples with different third values, holding the domain fixed. The priority differs (F test, p < .05) for 16 of the 171 pairs, against 30 when the domain varies with the triple held fixed (8.6 expected by chance in both).
  • Consideration versus choice: BWS asks which of three actions deserves most and least consideration, and P3 asks which of two actions to choose. The edges built from considerations and from choices agree (Table 4).
  • Main findings under P3: Between-model differences (§5.1) carry over: of the 165 cases where two of the four models differ by at least 20 points in a value pair's win rate under BWS, 148 keep their sign under P3. All 30 edges of the shared boundary (§5.2) keep their direction in each model. In these four models, holding the triple fixed, the priority differs across domains (F test, p < .05) for 27 of the 171 pairs under BWS and 35 under P3, against 8.6 expected by chance. The two sets share 17 pairs, against 5.5 if they were unrelated, and BWS gives 30 with all 22 models. The domain shifts agree between the two protocols (r = .82 over pair-by-domain cells). A cell's shift is the pair's win rate in that domain minus the average of the same triples across domains.

We will add this edge-level comparison of BWS and P3 to the appendix of the revised version.

Reviewer_wbzZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

this work proposes ValueGraph, a graph representation of how LLMs prioritize competing values. The authors use LLMs to build a dataset of value conflict scenarios and use it to measure the tradeoffs made by LLMs. They find that LLMs with similar overall value rankings can resolve specific tradeoffs differently, and these priorities shift with context. They conclude that alignment targets should specify priorities for each context.

Critical strengths

this work compares LLMs at the level of individual value tradeoffs rather than overall rankings. Across a broad set of models, this reveals differences that rankings alone would miss.

Critical weaknesses

1.The entire dataset is generated by a single model: the results may reflect that model's framing of value conflicts rather than the values of the models being evaluated.

Human validation does not address this concern, because validators only review items that have already been accepted and people tend to agree with the information presented.

2.Each value triple has at most one scene per domain, so domain effects cannot be separated from item effects. About half of the observed reversals would be expected by chance.Yet the central recommendation of context specific alignment rests on these results.

Questions for authors

Could you provide more details on the human validation, including how the validated items were sampled and how the validators' own choices compare with those of the models?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer wbzZ (1/3)

Thank you for highlighting the strengths of our trade-off-level comparison across a broad set of models. Below, we respond to each weakness and question.

1. (W1) Scenarios Generated by a Single Model

W1. "The entire dataset is generated by a single model: the results may reflect that model's framing of value conflicts rather than the values of the models being evaluated."

We tested this with a new experiment that does not rely on human review of accepted items. To see whether the dataset as a whole carries its generator's framing, other models redid the steps where a generator can leave its mark: they re-ranked the domains as a check on domain matching, and wrote a new scenario and its three actions for every rebuilt cell. The prompts, schemas and acceptance gates were the same. Every rebuilt scenario set was answered by 8 of the 22 evaluated models (2 Claude, 2 GPT, 2 Gemini, DeepSeek and Qwen).

Key finding: Edge directions and model differences hold on other generators' scenarios. Of the 962 edges the paper retains for these eight models, the other generators' 1,791 scenarios keep 926, reverse 20 and leave 16 tied (925, 21 and 16 on the original generator's 1,194 redraws). All 30 edges of the shared boundary (§5.2) keep their direction on every set. Of the 446 cases where two of these models differ by at least 20 points on a value pair in the paper, 417 keep their sign (412 on redraws). Other models largely agree with our domains, and no model family shifts significantly on its own scenarios.

Scenario generation. We rebuilt 620 cells balanced over values and value pairs (a cell is one value triple in one domain, holding one scenario in the original dataset). DeepSeek V4 Pro, Gemini 3.1 Pro and GPT-5.6 Sol each wrote a new scenario and its three actions per cell. Claude Sonnet 5, the original generator, wrote two more (redraws 1 and 2), which show how much results move when only the scenarios change. The results use the 597 cells valid in every set: 1,791 scenarios from the other generators and 1,194 redraws.

Domain matching. Four other models re-ranked the eight domains for 280–483 triples each. The domain we used is among their top three in 75–76% of cases and among their top five in 93–95%, against 37.5% and 62.5% by chance; Sonnet 5 repeating the task gives 82% and 97% (Table 1).

Table 1. Domains we used that fall in each model's top three and top five (%).

ModelIn its top threeIn its top five
DeepSeek V4 Pro7593
Gemini 3.1 Pro7595
GPT-5.6 Sol7593
Qwen 3.8 Max7695
Sonnet 5, repeated8297

Same-model effect. Prior research found that language models favor text they wrote themselves [1], so models might shift on scenarios written by their own family. For each of the 19 values, we took the change in its win rate (how often the value takes precedence, averaged over its 18 pairs) between two generators' scenarios. We computed it in models of a generator's family and in models of families that wrote neither set. z is the difference between these two changes divided by its standard error. Its largest absolute value is 1.65 (DeepSeek), 2.28 (Gemini), 2.53 (GPT) and 3.03 (Claude), below the Bonferroni-corrected 5% threshold of 3.33. No model family shifts significantly on its own scenarios.

Overall generator effect. Gemini's scenarios change the edges as little as a redraw does; GPT's and DeepSeek's change edge strengths more (Table 2). Retained edges are those that pass the paper's robustness checks (§5), counted when the set alone gives the same direction. One set alone keeps fewer than the 926 and 925 above, which pool three and two sets. Graph distance is the cosine distance between edge vectors. Replacement is the mean change in a value's win rate when these cells (a fifth of the dataset) are replaced by the set's scenarios.

Table 2. Each rebuilt set compared with the original set (eight models pooled, 597 cells valid in every set).

MeasureRedraw 1Redraw 2Gemini 3.1 ProGPT-5.6 SolDeepSeek V4 Pro
Rank correlation of value win rates.972.984.974.935.888
Retained edges keeping direction (of 962)882892885851798
Correlation of edge strengths (r).89.91.89.82.79
Graph distance.177.150.176.246.266
Replacement0.75%0.77%1.16%1.40%1.71%

We will add these experiments to the revised version.

AuthorsOpenReview ↗

Responses to Reviewer wbzZ (2/3)

2. (W2) Domain Effects versus Item Effects

W2-1. "Each value triple has at most one scene per domain, so domain effects cannot be separated from item effects."

In the paper, domain effects are estimated on value pairs (edges), not on triples, and a pair has a median of 6 scenarios per domain, each from a different triple. Following the best–worst scaling design of human value research [2, 3], one answer to a three-value scenario yields three pairwise decisions, so each pair has 41–51 scenarios (median 49) in 5–8 domains (median 7). A reversal is counted only between domains with at least three scenarios for the pair (every pair has at least four such domains), and only if it persists when any single scenario is deleted (§5.3), so no single item decides it.

To test this, we had the same generator, Sonnet 5, write two more scenarios independently for the same cells: with the item replaced, the domain effect remains. Three other generators add one each, so 597 cells hold six scenarios. If domain effects were item effects, adding scenarios would average them away. Instead the conclusion of the original dataset holds as scenarios are added (Table 3): with the original scenario, 25 of the 144 domain shifts are significant (7 expected by chance) and 42 of the 51 shifts significant in the rest of the original dataset keep their direction (half expected by chance); with the three Sonnet 5 scenarios, 29 and 42; with all six, 36 and 44. On these 597 cells, with the three Sonnet 5 scenarios as replicates, an F test of between-domain against within-cell variation is significant (p < .05) for 30 of the 171 pairs in the eight models (8.6 expected by chance). Ten remain after Benjamini–Hochberg correction for the 171 tests (false discovery rate 5%).

In Table 3, a domain shift is a value's win rate in a domain (how often it takes precedence within its pairs, over the eight models) minus the same pairs' win rate in the other domains; it is significant when its leave-one-triple-out jackknife z is at least 1.96. The reference shifts are computed on 2,193 scenarios of the original dataset outside these cells.

Table 3. Domain shifts on the 597 cells as scenarios are added to each cell.

Scenarios per cellSignificant shifts (of 144)Significant reference shifts keeping their direction (of 51)
1 (original dataset)2542
3 (+ 2 by the same generator)2942
6 (+ 3 by other generators)3644

W2-2. "About half of the observed reversals would be expected by chance."

The reversals exceed what chance produces, and the domain effect behind them recurs on new scenarios. With domain labels permuted among each pair's scenarios, only item variation remains, and it yields 121 persistent reversals (95% range 87–159); the paper observes 242. A chance pattern would not recur on new scenarios. The domain effect does. Domain differences computed on disjoint halves of each pair's third values correlate at ρ = .36, against .00 under permutation (App. G.3). The domain contrast behind the recommendation keeps its direction on left-out triples and on all five newly written scenario sets (W2-3 below).

W2-3. "Yet the central recommendation of context specific alignment rests on these results."

The recommendation rests on the 60 pairs between a Conservation value and an Openness or Self-transcendence value [4], taken together. The probability that the Conservation value takes precedence, relative to its mean over domains, rises by 12.5 points in Health and falls by 10.7 in Personal Development, with the same sign in all 22 models (§6). The Section 6 intervention compares prompts on the same scenarios, so item effects cancel there. A domain effect, unlike an item effect, should carry over to scenarios not used to estimate it, and this one does:

  • With one triple left out at a time, the same two domains are selected in 598 of 600 fits and differ by 18.5 points on the left-out triples.
  • On the five newly written scenario sets, the Section 6 contrast, Health minus Personal Development, is positive on all five (+0.3 to +22.5 points; pooled +8, 95% CI −1 to 17; Health holds 16 of the 597 cells). A broader contrast with more cells is also positive on all five: pooled +17 points (95% CI 6 to 28), 1 point below the +18 of the original scenarios (Table 4). Both were fixed before any new scenario was answered.

Table 4. Conservation's win rate over Openness values in Material Conditions and Health minus that in Personal Development and Environment, in points (597 cells).

Scenario setRedraw 1Redraw 2DeepSeek V4 ProGemini 3.1 ProGPT-5.6 SolFive rebuilt sets pooled
Difference from the original (+18)+4−13+4+8−11−1 (95% CI −12 to 9)

That value priorities shift with context is also found in prior research on people [5, 6] and on language models [7].

We will add this replicate analysis to the revised version.

AuthorsOpenReview ↗

Responses to Reviewer wbzZ (3/3)

3. (Q1) On Sampling & Human–Model Comparison

Q1. "Could you provide more details on the human validation, including how the validated items were sampled and how the validators' own choices compare with those of the models?"

  • Sampling: The 48 validators each reviewed 8 scenarios, covering 128 distinct scenarios with 3 independent validators per scenario (384 scenario evaluations; 1,152 action- and pair-level evaluations). The 128 scenarios were drawn uniformly by domain (16 per domain) from the 2,791-scenario dataset, spanning all eight domains, all 19 values, and 159 of the 171 value pairs.
  • Validators' vs. Models' Choices: Before seeing any quality-evaluation questions, each validator locked in a "most" and "least" consideration choice, fixing three pairwise decisions per scenario. On the same 128 scenarios, a model agrees with a validator about as often as two validators agree with each other (61.4% vs. 64.8% of pairwise decisions; range across the 22 models 58.1%–64.5%). This matches human value research: priorities vary mainly between individuals [8], while the average hierarchy is widely shared [9].

Table 5. Validators' own choices vs. model choices on the same 128 scenarios.

Analysis Scope & ComparisonMetric ValueSample Size (N)
Human – Human (two validators, same scenario)64.8%1,152 pairs
Human – Model (mean of 22 models; range)61.4% (58.1–64.5%)1,152 pairs

We hope our responses address your concerns, and we thank you again for your thoughtful feedback and questions, which have helped us improve the rigor and completeness of our paper.

References

[1] LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS, 2024

[2] Some Probabilistic Models of Best, Worst, and Best–Worst Choices. Journal of Mathematical Psychology, 2005

[3] Testing and Extending Schwartz Refined Value Theory Using a Best–Worst Scaling Approach. Assessment, 2019

[4] Refining the Theory of Basic Individual Values. Journal of Personality and Social Psychology, 2012

[5] Value Differentiation in Adolescence: The Role of Age and Cultural Complexity. Child Development, 2012

[6] Revisiting the Trans-Situationality of Values in Schwartz's Portrait Values Questionnaire. Quality & Quantity, 2019

[7] Stick to Your Role! Stability of Personal Values Expressed in Large Language Models. PLOS ONE, 2024

[8] Whence Differences in Value Priorities? Individual, Cultural, or Artifactual Sources. Journal of Cross-Cultural Psychology, 2011

[9] Value Hierarchies across Cultures: Taking a Similarities Perspective. Journal of Cross-Cultural Psychology, 2001

Comments on the paper

2 comments

We thank all reviewers for their feedback, which helped us to clarify the paper's argument (points 1 and 2) and strengthen its validation (points 3 and 4). We have uploaded a revised manuscript that incorporates the analyses and clarifications from our responses. Additions are marked in blue, and the main text now uses the additional page.

1. The Graph Representation at the Subgraph Level.

We further clarify how the graph representation connects a model's trade-offs into subgraphs and show that the two decisions in alignment evaluation (Sec. 6) are made at this level.

  • Sec. 6, Sec. 7. More explicit statement of the subgraph level and of what it adds to pairwise estimates with robustness analysis. (Reviewer L4J9 W1 and Q1, follow-up comments of 12s3 and L4J9)
  • Sec. 6, Appendix G.2, Table 21. New comparison showing that edges across the shared boundary are retained in every model, whereas those among the six favored values are retained in under a third of cases. (Follow-up comments of 12s3 and L4J9)
  • Sec. 6, Appendix G.6. Clarification that ValueGraph supplies the two domains of the alignment target and the target policy specifies only which value prevails in each. (Reviewer L4J9 Q3)

2. Relation to Prior Work.

  • Sec. 7, Appendix B.3. The revision makes clearer what ValueGraph contributes beyond ConflictScope, Priority Graph and the four works suggested by Reviewer 12s3, now also in the main text. (Reviewers 12s3 W1, L4J9 W1)

3. Edge-Level Validation across Protocols and Generators.

The new experiments validate the retained priorities at the edge level when the protocol changes and when other generators write the scenarios.

  • Sec. 4.3, Appendix F.5, Table 14. The protocol comparison now reaches the edge level, and almost every edge retained under BWS keeps its direction when the graph is rebuilt from pairwise choices. (Reviewers 12s3 Q2, L4J9 W2)
  • Sec. 4.3, Appendix F.6, Tables 15–16. On new scenarios written by three other generators, the models keep the direction of nearly all their retained edges and of every edge of the shared boundary. (Reviewers wbzZ W1, 12s3 Q3)
  • Sec. 4.3, Appendix F.7, Tables 17–18. On the new scenarios, score-implied priorities that the graph retains rarely reverse, whereas the others reverse far more often. (Reviewer 12s3 W2, Q1)

4. Domain Effects and Item Effects.

New experiments with replicate scenarios validate that the domain shifts in models' priorities are domain effects and not item effects.

  • Sec. 5.3, Appendix G.7, Table 25. With replicate scenarios in each cell, the significant domain shifts stay far above the number expected by chance. (Reviewer wbzZ W2)
  • Sec. 6, Appendix G.7. The Health–Personal Development contrast of Section 6 stays positive on all five new scenario sets. (Reviewer wbzZ W2)
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers and the Area Chair for their time, constructive feedback, and continued engagement throughout the discussion. ValueGraph contributes a dataset built from value triples across eight real-life domains, whose 2,791 well-validated scenarios cover all 171 pairs of Schwartz's 19 values, and a graph representation whose edges show which value takes precedence in each trade-off, how robustly, and in which contexts, and at the subgraph level what an alignment target must specify.

We are glad that all three reviewers affirm the paper's core claim. Evaluating individual trade-offs "reveals differences that rankings alone would miss" (wbzZ), identifies "a meaningful limitation of global value rankings" (L4J9) and is "an important direction" (12s3).

  • Reviewer wbzZ has not yet replied. W1 and W2 are answered by new experiments with newly written scenarios (point 3), and the robustness results on them are acknowledged by Reviewers 12s3 and L4J9.
  • Reviewer 12s3 concludes that "most of my concerns have been addressed" and "the dataset and empirical findings are useful enough to maintain my positive score".
  • Reviewer L4J9 has not yet replied to our subgraph analysis for the follow-up question (point 1), acknowledged by Reviewer 12s3, who asked the same question ("the comparisons with related work and the new subgraph analysis make the contribution clearer"). The reviewer acknowledged that the additional experiments "substantially address my validation concerns" and the Sec. 6 clarification "helps distinguish the domains identified by ValueGraph from the preference directions specified by the target policy".

1. What the Graph Representation Adds.

  1. Each edge shows whether its priority is robust. Reviewer 12s3 asked what ValueGraph captures beyond scalar profiles (W2, Q1). On new scenarios, score-favored priorities reverse far less often where the graph retains them (Sec. 4.3). The reviewer acknowledged this with "I appreciate the new robustness results".
  2. Read jointly, the edges inform two decisions in alignment evaluation. Reviewers L4J9 (W1, Q1, follow-up) and 12s3 (follow-up) asked what the graph representation adds beyond pairwise estimates. We responded that with robustness analysis these are its single edges, and clarified the subgraph level (Sec. 1, 6).
    • (a) Explicit preference judgments: Our new comparison (App. G.2) finds the 30 edges across the shared boundary robust in every model and the 15 among the six favored values in 92 of 330 cases. An alignment target requires explicit preference judgments on these 15 edges, singled out only by the boundary.
    • (b) Contexts to distinguish: Reviewer L4J9 asked which parts of Sec. 6 derive from ValueGraph (Q3). We clarified that it identifies the domains, and stated the target on a 60-edge subgraph with one preference direction per domain (App. G.6).

2. Relation to Prior Work.

ValueGraph measures what scores and rankings leave open. Reviewers 12s3 and L4J9 (W1) asked what ValueGraph adds beyond prior work. Separate scores do not determine which of two competing values gives way, and a ranking does not show which trade-offs are resolved robustly. ValueGraph measures every trade-off as an edge with its own robustness and domain dependence. Our comparison left Reviewer 12s3 "more convinced by the differences from the related works" (Sec. 7).

3. Validity of the Measured Priorities.

  1. The retained priorities hold on other generators' scenarios. Reviewers wbzZ (W1) and 12s3 (Q3) asked whether the results reflect one generator's framing. On other generators' scenarios, eight models keep the direction of nearly all retained edges (Sec. 4.3).
  2. The agreement between protocols extends from rankings to edges. Reviewers 12s3 (Q2) and L4J9 (W2) asked whether pairwise priorities agree across protocols. Rebuilt from pairwise choices, four models' graphs keep the direction of almost every edge retained under BWS (Sec. 4.3). Both experiments "strengthen the evidence for the robustness of the reported pairwise priorities" (L4J9).
  3. Replicate scenarios confirm the domain shifts as domain effects. Reviewer wbzZ asked whether they are item effects (W2). With more scenarios per cell, item effects would average away, yet more domain shifts are significant (Sec. 5.3).
  4. Models agree with validators about as often as two validators do, which is what balanced trade-offs should give. We added the comparison that Reviewers wbzZ (Q1) and 12s3 (Q3) asked for (App. E.4, G.2).

We believe ValueGraph provides a valuable foundation for evaluating how models resolve value trade-offs and determining what alignment targets must specify, with a dataset covering every value pair and a graph representation connecting them. We appreciate the reviewers' feedback, which helped us to sharpen this contribution and strengthen its evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.