acceptodds
Under review as a conference paper at ICLR 2027

Causal Consequence Invariance: Do LLM Causal Predictions Depend on Downstream Preferences?

Abstract

Large language models are increasingly used to produce predictions and analyses that may influence downstream decisions. In these scenarios, the user requesting a prediction may prefer one outcome over another. If such preferences change the model's estimate while the underlying causal evidence remains unchanged, then the prediction is less trustworthy since it no longer reflects the evidence alone. We construct 48 controlled structural causal model scenarios with exact interventional ground truth spanning confounding, collider bias, mediation, and no-adjustment settings. We find that across Claude Sonnet 5 and GPT-5, the mean absolute difference between opposing mild-preference conditions is only 0.053 and 0.004 percentage points, respectively. We find no detectable systematic variation in preference sensitivity across causal structures, and stronger stated incentives do not produce a practically meaningful monotonic response. Preference-induced variation is also comparable in magnitude to variation produced by causally irrelevant context. We further test whether this stability persists when the causal evidence becomes less precise and when abstract variables are replaced with semantically meaningful framings. Lower evidence precision produces a statistically detectable but practically negligible increase in absolute preference sensitivity for GPT-5, without a directional shift toward the preferred outcome; Claude shows no detectable increase, and semantic framing produces no detectable increase for either model. These results suggest that, in this controlled benchmark, both models' causal estimates are behaviorally invariant to the tested downstream preferences. More broadly, this work provides a practical way to evaluate preference sensitivity in the causal predictions of future models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Safety, LLM Evaluation, Causal Inference, Scientist AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JyACOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether downstream stakeholder preferences change an LLM's numerical causal estimate when the causal evidence is fixed. It operationalizes a behavioral analogue of consequence invariance using 48 binary structural-causal-model scenarios across four families: chain, confounder, collider, and mixed network. Each scenario has an exactly computed interventional answer and deterministically constructed observational counts. Matched prompts vary whether an external stakeholder prefers a higher or lower reported probability, with neutral prompts and three paraphrases providing additional comparisons. Claude Sonnet 5 and GPT-5 are evaluated in the core benchmark, secondary prompt controls, and stress tests involving evidence size and semantic framing. The main result is a very small mean absolute difference between opposing mild-preference conditions: 0.053 percentage points for Claude and 0.004 for GPT-5. The study uses scenario-cluster confidence intervals, directional analyses, and practical-equivalence criteria. The controlled results are informative; my concerns are whether the chosen contrasts fully measure the stated behavioral property and whether the experimental record supports reproducible interpretation of these very small effects.

Critical strengths

S1 The matched design isolates a clear scientific question and separates preference sensitivity from causal competence. The causal graph, evidence, intervention, and stakeholder identity are fixed within a matched comparison, while the preference statement is explicitly constructed to provide no additional causal information. The SCM-based ground truth avoids reliance on an LLM judge and allows direct accuracy checks. The confounder and network families include cases where naive observational reasoning can reverse the effect sign, so the benchmark tests more than simple association calculations. This is a useful extension of preference-sensitivity evaluation to quantitative interventional reasoning. A reliable negative result here would be valuable: it would identify a setting in which contemporary models resist a particular source of contextual contamination rather than merely document another failure case. S2 The analysis takes several important precautions for interpreting a null result. The paper resamples whole scenarios, preserving dependence among paraphrases and preference conditions, rather than treating every prompt as an independent experimental unit. It also uses practical-equivalence bounds, including a tighter 0.1-percentage-point check, instead of equating a nonsignificant test with proof of invariance. Directional shifts, an irrelevant-context comparison, causal-competence checks, and core tail statistics provide complementary diagnostics. The authors retain the isolated semantic-role error rather than discarding it and explicitly distinguish observable single-turn stability from the stronger training-level property proposed for Scientist AI. These choices strengthen the limited conclusion that the tested prompt manipulations usually produce very small numerical changes in this benchmark.

Critical weaknesses

W1 The main contrast measures disagreement between opposite preference directions, which is narrower than invariance across all preference conditions. The primary absolute difference compares the high-preference and low-preference answers. Both could agree while moving substantially relative to the neutral answer, for example if any stakeholder-preference statement changes how the model interprets the problem. The paper does examine neutral-referenced shifts and CCIdev for the core experiment, so this is not an entirely missing analysis. However, the generalization results are primarily presented through the opposing-preference contrast, and the all-condition neutral-referenced tails are not reported with comparable completeness. The industrial-framing example in Appendix D.2 makes this distinction concrete. A GPT-5 neutral response misinterprets causal roles and has approximately 12.7 percentage points of error, while the paper notes that this observation does not enter the primary opposing-preference metric. The response is appropriately retained, and a single anomalous generation does not establish preference-induced distortion. Nevertheless, small disagreement between the two directional prompts cannot establish that all three matched answers are stable. The actual neutral/positive/negative triplet and repeated responses would distinguish a stochastic reasoning failure, a shared contextual shift, and a systematic preference effect. This matters because Equation 3 defines invariance between preference conditions generally, and the paper's interpretation of the semantic stress test goes beyond just agreement between the two opposing directions. I would like the complete neutral-anchored results before treating that extension as evidence for the full stated behavioral property. W2 The benchmark's ability to probe susceptibility, and the reproducibility of the tiny numerical effects, need stronger support. The supplied graph is explicit, and the observational tables are deterministic conversions of analytic probabilities. Reducing their nominal sample size changes implied statistical precision, but largely preserves the same plug-in calculation apart from integer rounding. Appendix D.1 verifies uncertainty for a conventional estimator, which is useful, but it does not establish that the LLM treats the smaller table as an ambiguous inference problem. Indeed, the authors correctly note that the increased error under smaller sample sizes can be caused by discretization. Thus the stress test supports a specific evidence-size robustness result; it is less informative about whether preferences become influential when the model itself is uncertain. The third-person funding statements and stylized semantic relabelings also leave open whether the manipulation engages preference-sensitive behavior in this setting. Synthetic tests of the analysis functions do not answer that behavioral question. This limitation is compounded by the generation record. Each unique prompt is reportedly queried once under provider defaults, with no same-prompt repetition baseline, exact model snapshot identifiers, run dates, or full inference configuration. A scenario bootstrap estimates variability across the collected scenarios; it does not separate sampling variation from prompt-induced changes for a fixed scenario. The irrelevant-context control is useful but changes the input as well, so it does not replace that baseline. There is also an unresolved relationship among the core prompts, the medium-evidence prompts, and the abstract-framing prompts, which appear to share scenario, evidence, preference, and paraphrase settings but have slightly different summaries. This could have an innocent explanation in rendering or independent generation, but it needs to be documented. Without the frozen manifests and raw responses, a benchmark whose main effects are measured at hundredths or thousandths of a percentage point is difficult to audit or reproduce. These issues limit the strength and practical significance of the current conclusions, even though they do not negate the observed small differences.

Questions for authors

Q1 Coverage of the behavioral property — analysis of existing outputs. Please report neutral-to-positive and neutral-to-negative absolute deviations, CCIdev, and their upper tails for the uncertainty and semantic-framing experiments, not only for the core benchmark. For the industrial confounder_10 example, what are the exact three matched answers? Would the full all-condition metric still support practical equivalence at 1 and 0.1 percentage points? This is central to whether the semantic extension tests the property defined in Equation 3. Q2 Generation variability — a limited replication. On the existing diagnostic subset, could you repeat identical prompts under a fully specified configuration and compare within-condition variability with the between-preference contrasts? Repeating all three conditions for the semantic-role-error cell would be especially informative. This would help determine whether the remaining numerical differences and occasional large neutral shifts reflect sampling, contextual effects, or preference direction. I am asking for a bounded diagnostic, rather than a new large-scale benchmark. Q3 Manipulation sensitivity — clarification and a targeted control. Is there evidence that the tested preference statements can affect model behavior under a matched condition in which preference sensitivity is expected or independently observed? A small behavioral positive control, or a matched first-person stakeholder phrasing on the existing subset, would help interpret resistance to the current third-person manipulation. For the evidence-size test, do the models' own uncertainty reports or reasoning show that they recognize the reduced precision? If not, please narrow the interpretation to stability under the displayed count transformations. Q4 Prompt reuse and model configuration — clarification. Are the medium-precision prompts in Appendix D.1 and abstract prompts in D.2 byte-identical to each other or to the corresponding core prompts? If so, were generations reused or repeated, and how does this relate to the claim that each unique prompt was queried once and to the 1,980-generation count? Please provide the hash-to-generation mapping, exact model identifiers, execution dates, system instructions, and relevant sampling/reasoning settings. This would materially improve confidence in the reported effects. Q5 Reproducible benchmark and incremental insight — clarification. Can the frozen scenario bank, actual parameter-generation and acceptance rules, all paraphrase templates, raw responses, and analysis scripts be provided anonymously during review? Appendix A describes selection criteria qualitatively and reports the resulting ranges, but does not supply all numerical rules or instantiated cases. Also, please clarify the incremental claim relative to earlier studies that vary a user's liking or disliking while holding task content fixed. I see the important extension in exact SCM-grounded interventional estimates; specifying this contribution more precisely would strengthen the paper's significance argument.

Reviewer_eEm2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether LLMs change their causal predictions depending on who benefits from the reported answer, even when the underlying evidence is unchanged.

For example, an LLM estimates an employee’s chance of completing a task successfully if they take a training program. One prompt ties the requesting team’s bonus to a high reported probability; another ties it to a low one. The data and causal relationships stay the same.

The authors compare Claude Sonnet 5 and GPT-5 across 48 synthetic scenarios with known answers. Additional tests vary the wording, financial stakes, amount of evidence, and meanings attached to the variables.

The estimates barely change: the mean absolute difference between opposing mild preferences is 0.053 percentage points for Claude and 0.004 for GPT-5, with no systematic shift toward the preferred answer. The contribution is a controlled benchmark and evidence of stability under the tested incentives, in tasks where the causal graph, outcome, and question are already specified.

Critical strengths
  • S1. An interesting and important research question. Whether predictions change depending on who benefits from the reported number matters when LLMs inform research, policy, and organizational decisions. Isolating the influence of these interests from agreement with a user’s stated beliefs is a useful contribution.

  • S2. Careful experimental control and validation. The study uses four causal structures with exactly computable answers and compares preference conditions while holding the evidence fixed. It distinguishes prediction accuracy from preference-induced changes and evaluates both statistical significance and practical effect sizes.

Critical weaknesses
  • W1. Limited explanatory insight. Both models are already highly accurate on tasks with supplied causal graphs and fixed queries. The robustness tests largely reproduce stability without explaining why it occurs or when it would break. Although the authors acknowledge this narrow scope, the findings offer limited insight into judgments involving uncertain causal assumptions or competing interpretations.

  • W2. Unclear added contribution of the causal setting. Prior work examines dishonest reporting for users’ benefit (Köbis et al., 2025) and numerical estimates that change with the consequences of the answer (Betley et al., 2026). The exact causal answers and the manipulation of preferences alone provide useful control, but it remains unclear what the causal setting adds. Comparing with non-causal numerical tasks matched on difficulty and baseline accuracy, under the same incentive manipulation, would clarify whether the findings reveal anything specific to causal estimation.

Questions for authors
  1. [Added contribution] What does the causal setting add beyond testing resistance to distorted numerical reporting? Do you have existing comparisons with non-causal numerical tasks under the same preference manipulation, accounting for difficulty and baseline accuracy?

  2. [Interpretation of the null result] Do you have a behavioral control showing that these preference statements can shift actual model estimates? Appendix E validates the statistical analysis using injected effects, but such a control would help interpret the near-zero response observed here.

Reviewer_qJkUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

A benchmark paper that asks whether telling an LLM that a stakeholder has a stake in the consequences of a causal estimate affects that LLM's causal estimate. Provides a benchmark contains 48 binary SCMs scenarios with interventional ground truth answers spanning different causal DAG motifs (chains, confounding, colliders, etc.). Matched prompts vary whether a stakeholder benefits from a higher, lower, or neutral reported estimate, with three paraphrases. Secondary experiments test robustness of analysis by varying preference strength, irrelevant context, prompt position, stakeholder identity, evidence precision, and semantic framing.

Principal result is a negative result: Across 48 controlled problems, Claude Sonnet 5 and GPT-5 barely change their estimates.

Critical strengths

The benchmark and the negative results are useful. Testing across different DAG motifs, preference strength, irrelevant context, prompt position, stakeholder identity, evidence precision, and semantic framing lends confidence to those results.

Critical weaknesses

To be clear, I'm all for well-supported negative results.

Still, an evaluation covering two recent proprietary models from frontier AI labs limits what we learn about the conditions under which stakeholder preferences distort causal predictions.

Questions for authors

What exact model versions and reasoning configurations were used, including whether thinking was enabled and any reasoning budgets?

Why not do a systematic comparison including smaller models, earlier-generation systems, or models that are trained/fine-tuned in ways that might display preference sensitivity in causal predictions? If there is some architectural or training regime where this preference sensitivity shows up, then can follow up with a controlled experiment that confirms. This would strengthen the paper by telling the reader under what settings they are at risk for preference sensitivity. Even preliminary results of scan of smaller open source models here would be good. Would consider upgrading rating based on this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.