acceptodds
Under review as a conference paper at ICLR 2027

The Within–Across Gap: Behavioral Value Consistency of LLMs in Multi-Step Scenarios

Abstract

A common goal in research on LLM alignment with human values is to summarize a model with a compact value profile, e.g., a ranking of values, that predicts behavior in new social situations. To examine how such profiles relate to behavior, we focus on two aspects: behavioral consistency (the model makes consistent choice across same value tradeoffs) and explanatory power (how well a single profile accounts for observed behavior across scenarios). Existing elicitation methods, which typically rely on isolated single-turn prompts, provide limited evidence for jointly examining these aspects within and across scenarios. We introduce a benchmark of multi-step branching scenarios for evaluating behavioral value consistency and the explanatory power of compact value profiles, covering all 45 pairwise tradeoffs in the Schwartz value framework, with 10 scenarios per tradeoff. Each scenario is a decision graph in which a model makes a sequence of choices; every option is annotated with the values it promotes or sacrifices, and every decision node with the type of narrative pressure applied. Evaluating 13 open-source LLMs across four model families and three proprietary systems, we find that (1) within-scenario consistency is generally higher than cross-scenario consistency, (2) consistency generally increases with model scale, (3) a single latent value ranking accounts for part, but not all, of observed model behavior, and (4) within-scenario value-side flip rates vary across narrative pressure types. Overall, LLMs exhibit context-dependent value consistency, while fixed rankings provide informative but partial summaries of their behavior.

Primary area:
datasets and benchmarks
Keywords:
LLM, value consistency, behavioral evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BsjbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the behavioral value consistency of LLMs in multi-step decision-making scenarios. The authors construct a benchmark consisting of 450 branching scenarios based on 45 pairs of Schwartz's basic human values. Each scenario involves a sequence of decisions where the model must make trade-offs between competing values. The experiments cover multiple LLM families and sizes. The main finding is that models exhibit higher consistency within scenarios than across scenarios, with the gap generally decreasing for larger models. The authors further analyze whether model behavior can be explained by a one-dimensional value ranking and how different types of narrative pressure are associated with preference reversals.

Critical strengths
  • An interesting and relevant research question. Most existing evaluations of LLM value alignment focus on isolated questions or static preference rankings. This paper instead studies model decisions in multi-step scenarios, where previous choices and changing circumstances can affect subsequent decisions. I think this is a useful setting for studying LLM behavior beyond single-turn evaluations.

  • A reasonably comprehensive benchmark and experimental analysis. The benchmark covers 45 value pairs and 450 scenarios, and the authors evaluate a broad range of models. The paper also goes beyond reporting consistency scores by analyzing value-ranking structures, preference reversals, and robustness to paraphrasing. These experiments make the empirical observations more convincing.

Critical weaknesses
  1. The within–across consistency gap may be partly influenced by the metric design. The two consistency metrics are calculated at different aggregation levels. Within-scenario consistency is computed from decisions within a single trajectory, while cross-scenario consistency pools decisions across multiple scenarios. Since both metrics use the majority preference proportion, shorter trajectories can naturally produce higher consistency scores, even without genuine scenario-dependent behavior. Although the authors include random baselines, I am not convinced that these fully address this issue. A more appropriate control would preserve the number of decisions in each trajectory while assuming a fixed value preference across scenarios.

  2. The interpretation of cross-scenario inconsistency is not sufficiently justified. Different scenarios involving the same value pair may have very different consequences, risks, and trade-off intensities. For example, prioritizing security over benevolence in one scenario but making the opposite choice in another does not necessarily imply an inconsistent value preference. The decisions may simply reflect different circumstances. The current annotations mainly capture the direction of value preferences, rather than their relative importance or the magnitude of the trade-offs. The authors acknowledge this limitation, but I think it directly affects the main conclusion. The two human participants also exhibit a similar within–across gap, which raises the question of whether this pattern necessarily indicates a meaningful deficiency in LLM behavior. I also find the narrative-pressure analysis somewhat descriptive. Different pressure types are associated with different reversal rates, but it is unclear whether these differences are caused by the pressure itself or by other properties of the scenarios.

Questions for authors
  • Regarding the consistency metrics: Have the authors considered a baseline where each value pair has a fixed preference probability, and decisions are sampled independently using the same trajectory lengths as the benchmark? This could help estimate how much of the within–across gap arises from the metric definitions.

  • Regarding scenario comparability: How do the authors ensure that different scenarios involving the same value pair have comparable trade-off intensities? Could differences in consequences or risk levels explain part of the observed gap?

  • Regarding narrative pressure: Would it be possible to compare matched decision points with and without a particular pressure type? Such a comparison would help clarify whether the reported reversal patterns are actually related to the narrative pressure rather than other contextual factors.

3 comments

Thank you for your thoughtful review and for suggesting a specific control for the metric design. We ran it as you described. We address each point below.

W1 & Q1. Yes, this is the baseline we ran (R2 asked the same). For each value pair, we now fix its preference rate and the trajectory lengths and then sample each choice (1,000 samples). The excess (observed gap - null gap) is still significant in all 19 models.

ModelRaw gapNull gapExcess [95% CI]p < 0.05
Qwen3-4B0.1520.1110.041 [0.022, 0.062]✓
Qwen3-8B0.1090.0890.020 [0.002, 0.039]✓
Qwen3-14B0.1130.0870.027 [0.006, 0.048]✓
Qwen3-32B0.1160.0780.038 [0.020, 0.057]✓
Gemma-3-4B0.1380.1090.030 [0.015, 0.045]✓
Gemma-3-12B0.0980.0810.017 [0.000, 0.034]✓
Gemma-3-27B0.1250.0820.043 [0.023, 0.065]✓
Phi-3-mini0.1430.0960.047 [0.028, 0.064]✓
Phi-3-small0.1200.0790.041 [0.021, 0.062]✓
Phi-3-medium0.1220.0850.036 [0.015, 0.058]✓
Mistral3-3B0.1440.1080.035 [0.017, 0.055]✓
Mistral3-8B0.1330.0940.038 [0.021, 0.056]✓
Mistral3-14B0.0870.0650.022 [0.009, 0.037]✓
DeepSeek-V30.0960.0670.029 [0.012, 0.048]✓
Claude-Sonnet-4.60.0980.0640.034 [0.015, 0.054]✓
GPT-4o-mini0.1040.0870.017 [−0.001, 0.035]✓
GPT-4o0.1090.0790.031 [0.009, 0.053]✓
GPT-5.4-mini0.1070.0700.036 [0.017, 0.056]✓
GPT-5.40.0990.0600.039 [0.019, 0.061]✓
Mean0.1170.0840.03319/19

CIs are bootstrapped over value pairs; p is from the sampling null.

We will replace the raw-gap test in Table 1 with this analysis.

W2 & Q2. We do not match intensity across scenarios (there is no objective common scale for it across different stories). Actually we first scored each option's value intensity on a 1–3 scale, but annotator agreement on these scores was low as it's very subjective. So all analyses use only the direction (sign level). Instead, we make sure every option is a reasonable choice and its consequences are not exaggerated, so that no scenario collapses to one obvious option (a dedicated anti-collapse/balance item in our human evaluation: 4.90/5; Table 5, App. J).

We agree that different circumstances can lead to different choices. This is exactly our point. We do not claim that LLMs are flawed. We claim that a fixed value profile (a ranking or score), as used in prior work, cannot describe choices that depend on the scenario. Humans show the same gap, which supports this.

W2 & Q3. We agree this would be an interesting comparison, but our benchmark was not designed for it. In a branching story, the pressure is part of the plot, so changing it at one node changes the later nodes and the tree. Different subtrees follow different storylines, which makes a fair comparison difficult.

Thank you again for your thoughtful comments. We have added additional analysis and clarification. We would appreciate it if you could reconsider your assessment and score based on these responses, and would be happy to clarify any remaining questions.

Dear Reviewer Bsjb,

As the rebuttal phase is nearing its end, we would appreciate hearing whether our rebuttal has resolved the issues you raised. If any concerns remain, please let us know so that we can address them in the remaining time.

If our responses address your concerns, we would sincerely appreciate it if you could reassess your score.

Thank you for your time and thoughtful review.

Reviewer_tHWBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces 450 branching scenarios spanning all 45 pairs of ten Schwartz values to measure consistency within and across narratives. It reports results for 19 models in Table 1, a human quality audit of 90 scenarios, and paraphrase robustness checks. Within-scenario majority scores exceed pooled cross-scenario scores, while larger models in several families show greater consistency. Further analyses examine low-rank preference matrices, held-out prediction from value scores, and associations between narrative pressure and value-side changes.

Critical strengths
  1. The branching benchmark makes it possible to study repeated value tradeoffs while earlier choices change subsequent situations (Section 3). Coverage of 45 pairs, a stratified 90-scenario human audit, and the construction prompts in Appendices M–P give the dataset a substantive foundation. This is a useful evaluation setting beyond isolated choices.

  2. The robustness analysis separates full trajectory re-runs from fixed-history single-step paraphrasing (Appendix H). Across-model correlations for the consistency measures remain above 0.92 under both variants, and the option-level results expose residual wording sensitivity. These checks provide concrete support for studying more than isolated prompt wording effects.

Critical weaknesses
  1. The headline within–across gap lacks a directly calibrated null comparison (Section 4.1, Table 1). Averaging a majority fraction over short trajectories and computing it after pooling ten trajectories have different finite-sample biases. With equal lengths, the former is necessarily at least the latter. Even independent fair choices yield an expected gap of about 0.125 for ten four-choice trajectories; this is an illustrative calculation, not an estimate of this dataset's null. The paper uses unequal trajectory lengths and unweighted averaging, which additionally requires attention to weighting. Table 1 reports a raw intra-greater-than-cross test and compares each metric with shuffled assignments, but does not report whether the gap itself exceeds a length- and preference-matched null. Thus, its significance does not establish the magnitude of narrative-specific coherence.

  2. The ranking diagnostics do not identify the proposed score-difference structure (Section 5, Appendix G). C_ab = pi_a - pi_b implies rank at most two, but a rank-two skew-symmetric matrix can contain a three-way preference cycle and admit no such scores. Consequently, a small sigma_3/sigma_1 or rank-two reconstruction error is insufficient evidence for that structure. Appendix G's leading-singular-vector procedure also requires justification: a nonzero skew-symmetric logit matrix has paired singular values, and an unconstrained rank-one approximation is not an additive ranking fit. The Borda results do show predictive signal, but neither these diagnostics nor imperfect performance of the two estimators establishes the explanatory limit of a well-fitted value-score model. Both headline analyses therefore require substantive reanalysis.

Questions for authors
  1. Can you report the observed gap minus a null gap that preserves trajectory lengths and pairwise preference rates, with uncertainty, and clarify handling of zero/one tradeoff choices and choices favoring both values? Would the family-level conclusions remain?

  2. Can you fit the score-difference or option-utility model directly, evaluate it on graph-disjoint held-out folds, and compare it with the reported SVD-WR estimator? Please specify the existing fold construction and SVD sign/degeneracy handling. These analyses could materially change my assessment.

8 comments

Thank you for your careful reading and your concrete suggestions on the null model and the ranking analysis. They helped us make our analysis more rigorous. We address each point below.

Q1. For each value pair, we fix its preference rate (the overall share of choices for each value) and the trajectory lengths. We then sample each tradeoff choice independently from this rate (1,000 samples). The null gap is the mean gap (VC_intra − VC_cross) over these samples. Excess = observed gap − null gap.

ModelRaw gapNull gapExcess [95% CI]p < 0.05
Qwen3-4B0.1520.1110.041 [0.022, 0.062]✓
Qwen3-8B0.1090.0890.020 [0.002, 0.039]✓
Qwen3-14B0.1130.0870.027 [0.006, 0.048]✓
Qwen3-32B0.1160.0780.038 [0.020, 0.057]✓
Gemma-3-4B0.1380.1090.030 [0.015, 0.045]✓
Gemma-3-12B0.0980.0810.017 [0.000, 0.034]✓
Gemma-3-27B0.1250.0820.043 [0.023, 0.065]✓
Phi-3-mini0.1430.0960.047 [0.028, 0.064]✓
Phi-3-small0.1200.0790.041 [0.021, 0.062]✓
Phi-3-medium0.1220.0850.036 [0.015, 0.058]✓
Mistral3-3B0.1440.1080.035 [0.017, 0.055]✓
Mistral3-8B0.1330.0940.038 [0.021, 0.056]✓
Mistral3-14B0.0870.0650.022 [0.009, 0.037]✓
DeepSeek-V30.0960.0670.029 [0.012, 0.048]✓
Claude-Sonnet-4.60.0980.0640.034 [0.015, 0.054]✓
GPT-4o-mini0.1040.0870.017 [−0.001, 0.035]✓
GPT-4o0.1090.0790.031 [0.009, 0.053]✓
GPT-5.4-mini0.1070.0700.036 [0.017, 0.056]✓
GPT-5.40.0990.0600.039 [0.019, 0.061]✓
Mean0.1170.0840.03319/19

CIs are bootstrapped over value pairs; p is from the sampling null.

As the reviewer anticipated, most of the raw gap (0.084 of 0.117) is expected under the null. But the remaining excess (0.033) is still significant in all 19 models, so the gap between within- and cross-scenario value consistency still holds, our main conclusion remains unchanged.

Handling. Trajectories with no tradeoff choice (9.9%) are excluded. Those with one choice (9.0%) enter VC_cross only. Both/neither options (8–19% of choices) are excluded.

Family level. Larger models remain more consistent: VC_cross rises in all five families (+0.03 to +0.11). But the excess gap does not narrow with scale (changes of −0.013 to +0.015, none significant) anymore. We will revise this claim and replace the raw-gap test in Table 1 and Sec. 4.2 with this analysis.

Q2. We follow both suggestions. For the split, we use graph-disjoint folds: each fold holds out 2 of every value pair's 10 scenario graphs, and ties receive expected credit. For the fit, we estimate the score-difference model (least squares on logit win rates) and the option-utility model (conditional logit) directly.

EstimatorFoldsk=2k=3
Random–0.5000.333
SVD-WRnode (Table 8)0.6020.426
SVD-WRgraph0.6030.421
Bordanode (Table 8)0.6200.449
Bordagraph0.6170.438
Score difference (LS)graph0.6100.429
Option utility (conditional logit)graph0.6340.451

Mean held-out top-1 accuracy over 19 models. All estimators use sign-only value vectors: SVD-WR is recomputed with the sign-only win-rate matrix of Sec. 5, and LS is fit to observed pairwise win rates (share of a-vs-b tradeoff choices won by a).

Graph-disjoint folds change accuracy by ≤0.011, and on the same folds the best direct fit improves on SVD-WR by ≤0.031. Neither the split nor the estimator therefore explains the residual: even the best model misses 37% of two-option and 55% of three-option choices, consistent with the scenario dependence in Q1. Therefore, the conclusions made in the paper still hold.

Existing procedure. Table 8 used 5 random folds over pooled decision nodes. SVD-WR takes the leading left singular vector of the antisymmetrized logit win-rate matrix L, with its sign fixed to agree with the row sums.

On σ3/σ1. We agree that our σ3/σ1 analysis can absorb cycles into the "one-dimensional" part, since a cyclic matrix can also be rank 2. We therefore apply HodgeRank (Jiang et al., 2011) to the sign-only logit win-rate matrix (Sec. 5), which separates a ranking part from a cyclic part: the ranking accounts for 71–90% of the energy, while the remaining substantial cyclic component (10–29%) cannot be represented by any ranking.

ModelRankingCyclic
Qwen3-4B75.6%24.4%
Qwen3-8B77.7%22.3%
Qwen3-14B82.2%17.8%
Qwen3-32B86.1%13.9%
Gemma-3-4B71.3%28.7%
Gemma-3-12B87.3%12.7%
Gemma-3-27B81.9%18.1%
Phi-3-mini77.4%22.6%
Phi-3-small84.0%16.0%
Phi-3-medium86.1%13.9%
Mistral3-3B71.0%29.0%
Mistral3-8B85.0%15.0%
Mistral3-14B90.4%9.6%
DeepSeek-V388.1%11.9%
Claude-Sonnet-4.687.2%12.8%
GPT-4o-mini83.1%16.9%
GPT-4o82.7%17.3%
GPT-5.4-mini85.6%14.4%
GPT-5.483.4%16.6%
Mean82.4%17.6%

We will add these direct fits and the HodgeRank decomposition to Sec. 5, Tables 3/8 and App. G.

Thank you again for your valuable feedback. We have provided the requested additional analyses and clarifications, and would be grateful if you could reconsider your assessment and score in light of our responses. Please let us know if anything needs further clarification we would be happy to provide.

Reviewer_9FDJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors introduced a benchmark of multi-step branching scenarios for evaluating behavioral value consistency of large language models using the Schwarz value framework. They mentioned that their motivation for creating this work was in response to the gap in existing value-alignment benchmarks where researchers typically use single forced choice to evaluate the value orientation of language models, implicitly arguing that these benchmarks measure shallow value representations. To operationalize this multi-step branching scenarios, the authors modeled them as a decision graph through which the model makes decisions. They further leveraged a language model to create a synthetic benchmark and tested this benchmark on four open source families and nineteen variants of models within these families. The also tested their system on three closed source language model families. The results from their study showed that among other things that the performance of a model on value consistency correlates with the scale of the model.

Critical strengths

Strengths

  1. This paper is well motivated and is targeted at a gap that is long overdue to be resolved.
  2. The paper literature review section positions the paper well in terms of the gap it is trying to resolve
  3. The writing is clear and easy to follow
Critical weaknesses

Weaknesses

  1. The distance in the annotator agreement seems to be significant and overall not very high as per the Krippendorff's range in Appendix E which is between 0.318 to 0.688. What was the cause of this? what did the annotators agree on vs disagree?
  2. The human reviewers seem to have focused on plausibility of the scenarios and not on the actual representation of human values in the evaluation.
  3. The human baseline which is just n=2 is rather too small.
  4. The result in the paper was not so surprising. Could it be that the LLM generated the value scenarios that is within its distribution, which in turn made it easy for them to answer, such that larger models were more likely to have the answers within their distribution?
  5. The study instrument could have been made better. The reason for this is that, given that human value is inherently a human property and naturally generative and also considering that LLM already contain publicly available value-oriented sources, using LLMs to generate synthetic human value datasets is not innovative enough. The work could have explored more live sources and contemporary situations for this research.
  6. Essentially the research instruments involves using LLM to generate a synthetic benchmark that another LLM further tested on.
Questions for authors
  1. What was the cause of the variation in the annotation results and where did the annotators agree on vs disagree?
  2. Did the work explore other approaches for creating the multi-step human values benchmark and why did they settle on this approach?
3 comments

Thank you for your positive comments on our motivation, We address each of your concerns below:

W1 & Q1. The lower α values mainly come from a ceiling effect (App. E). α measures how well annotators separate good from bad items. When almost all items get 5, there is little to separate, so α drops even when annotators give nearly the same scores. This is why branch naturalness has the lowest α (0.318) but the highest mean (4.93, SD 0.28). Dimensions with more spread have higher α (value-assignment accuracy: 0.688).

W2. Our human evaluation does check value representation. Two of the six criteria are about values (Sec. 3.3, App. J): value-assignment accuracy (whether each option's value labels are correct) and anti-collapse/balance (whether both values are fairly represented).

W3. The purpose of the human baseline is to test whether the within–across gap is unique to LLMs, or also appears in individual humans. n = 2 is sufficient to show that this gap exists within some humans, but we agree that it does not allow us to make a general statement about humans. We will add this to the Limitations section.

W4. Only one of our 19 models is from the generator's family (Claude-Sonnet-4.6). Our findings also hold in the other 18 models. The one Claude model does not stand out either (VC_cross 0.744, between GPT-5.4-mini at 0.717 and GPT-5.4 at 0.748). Some LLM bias may still remain; we explain below why LLM generation was still needed.

W5, W6 & Q2. As our related work shows, existing value datasets are mostly single-turn and cannot track how choices change as a conflict evolves. Moving beyond this sets high structural requirements (multi-step branching trees, stories built around a value tradeoff, value conflicts that evolve across steps, and value labels for every option). No existing dataset meets them, so we turned to LLM generation. As described in Sec. 3.2, we build each scenario step by step, we tried other approaches first (generating a whole scenario in one pass, or expanding from existing story seeds etc), but dropped them because of low quality. Although the scenarios are LLM-generated, we still anchor them to real data (topics come from the everyday dilemmas in DailyDilemmas; Chiu et al., 2024).

Thank you again for your constructive feedback. We have provided further explanations addressing your concerns, and hope you would consider revisiting your assessment and score. If there are any remaining questions or points that need clarification, we would be happy to provide further details.

I thank the authors for engaging with the feedback. Some of my concerns have been addressed like W1 & 2. For the other concerns, your rebuttal seems to be framing it as a data engineering problem, where existing datasets don’t meet the structural requirements. But my point is about validity questions, like how do we know what we are measuring through this research. I will keep my score as is for now.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

We thank all reviewers for their thoughtful feedback. For ease of reference, we summarize below the main concerns raised and our corresponding responses:

Main concernReviewer(s)Response and current status
Benchmark construction and human validation9FDJExplained the ceiling effect on annotation agreement and clarified that the human evaluation covers both value-label accuracy and value balance. Further explained the grounding in real-world dilemmas and the role of the human results as individual-level references.
Whether the consistency gap reflects metric biastHWB, BsjbAdded a null baseline preserving trajectory lengths and pairwise preference rates. All 19 models retain a significant excess gap under the sampling-null test. Higher cross-scenario consistency in larger models remains supported.
Limits of fixed value scores in predicting actual choicestHWBAdded graph-disjoint evaluation and direct fits of the score-difference and option-utility models. The best prediction accuracies are 63.4% / 45.1% for two-/three-option choices, showing that these fitting method capture only part of the observed choice behavior across scenarios, leaving substantial residual prediction errors.
Distinguishing value rankings from cyclic preferencestHWBAdded a HodgeRank decomposition to assess how well additive value scores capture the observed preference structure. The ranking component accounts for 71–90% of the matrix energy, while 10–29% is cyclic, showing that fixed value scores capture the dominant preference structure but leave a substantial cyclic component can't explained by ranking.
Interpretation of cross-scenario variationBsjbClarified that our conclusion concerns how well a fixed value ranking describes choices across scenarios. Context-dependent variation can reflect changing circumstances and does not imply a behavioral deficiency.
Interpretation of narrative-pressure effectsBsjbExplained that the analysis characterizes associations between narrative pressure and preference reversals. Since pressure is embedded in branching storylines, matched causal comparisons would require a dedicated experimental design.

As the discussion period closes in less than 20 hours, we would greatly appreciate a brief follow-up from Reviewers tHWB and Bsjb on whether our responses address their concerns. We would also be grateful if they could consider updating their assessments in light of the additional analyses and clarifications. We are actively following the discussion and are ready to answer any final questions promptly.

Thank you again for your time and thoughtful consideration!

Best regards, Paper 34302 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.