acceptodds
Under review as a conference paper at ICLR 2027

Decorative Rationalization in Large Language Models

Abstract

Many modern uses of LLMs involve answering questions over information supplied at test time – diagnosing a patient from test results, reviewing a contract for risky clauses, or debugging code from an error trace. In these tasks, the model returns an answer along with rationale that people could judge to decide whether to rely on the answer. In this paper, we reveal that models produce decorative rationale, citing parts of the context as reasons for an answer that does not depend on them. We formalize this failure through the probability that removing a cited part leaves the answer unchanged, and estimate it through causal intervention on instruction-following and reasoning models. Our results reveal that much of the cited information is used decoratively. On average, removing a cited part leaves the answer unchanged 45 to 87% of the time. Decoration survives every condition we test. It persists when the model reasons before answering and when the model cannot answer without the context. It also persist when we strip the decorative context, because the model re-decorates with whatever remains. LLMs thus rationalize, creating a harmful illusion that may only be fixable at training time.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
faithfulness, rationalization, reasoning, large language models, causal intervention, context attribution, misalignment, trustworthiness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TxFyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This research investigates the frequency with a LLM providing the same response after the context referenced in the LLM's explanation has been removed. It was found that most of the answers were preserved after removing the context from the question-answering data sets. However, even though many of the answers were preserved, there was only a small improvement when the context was removed and the answers were regenerated.

Critical strengths

This study analyzed approximately 53,000 citations from both instruction following models and reasoning models. There were also evaluations using closed models and synthetic data.

Critical weaknesses

1)The main issue is that the authors have over-interpreted their metric as evidence of unfaithful rationalization. Answer preservation after deletion is a measure of dispensability; it does not indicate whether the original computation ignored the cited evidence.

2)The causal decomposition used by the authors is not justified. Answers provided correctly without context do not demonstrate that context was not used in the first place. The "prior knowledge" criterion used by the authors can also classify examples where the original answer was different or incorrect. The authors' metric also fails to capture substantive evidence use. Evidence may affect confidence levels or answer probabilities without affecting which answer is chosen.

3)The authors do not evaluate their proposed content replacement criterion. The permissive answer matching used by the authors may inflate the results. Subsets and supersets are classified as "SAME", which means that the loss of material answer content could be counted as evidence of non-use. The statistical guarantees provided by the authors do not account for measurement error. Since unvalidated LLM judges and context rewrites are used to generate the data, the confidence bounds cannot be relied upon to accurately detect actual rationale unfaithfulness.

4)The mitigation experiment conducted by the authors does not support their conclusion. Approximately 18-22% of relevant HotpotQA cases result in changed answers when joint stripping is applied. Therefore, failure of this single procedure does not demonstrate that inference-time mitigation is inadequate.

Questions for authors

How is a citation that has never been cited in the first place distinguished from a citation that was once cited but then replaced by a different path of reasoning after it was deleted?

How can you demonstrate that the original response did not use the context based on correctness alone (i.e., without considering the context)?

How would your findings be affected if you measured influence using answer probability instead of final-answer identity?

Will the reported rates still be comparable if you no longer classify subsets and supersets as equivalent answers?

Does re-decoration continue to occur when joint removals result in preserving the original answer, and what evidence supports extending this situation beyond the single mitigation studied?

Reviewer_9TMBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work studies a highly relevant problem among reasoning models and tests whether the parts of the input that an LLM cites in its rationale are the parts its answer depends on. It proposes the term decorativeness of a of a context part as the probability that removing it leaves the answer unchanged. With a thorough analysis of different types of queries and models, the work establishes different categories of decoratives and draw conclusions about how much a model relies on prior knowledge. Removing the decorative parts and regenerating the rationale lowers but does not eliminate decoration. The authors conclude that the behaviour likely requires a training-time fix.

Critical strengths
  • The problem is as it stands is well motivated and highly relevant. There are multiple threads of literature that point towards models that appear convincing through elaborate reasoning or hallucinations
  • The current framework tests the property of a model in itself and needs no ground truth, similar to faithfulness metrics in literature. Hence, it also scales well.
  • The empirical scope is broad with multiple types of queries and useful analysis pointing towards lack of dependency on context, or over-dependency on prior knowledge is an interesting analysis. These concerns are usually reported while training, so formalizing a framework that can test a model's dependency on prior knowledge is promising.
Critical weaknesses
  • One of the fundamental methodological flaws is in what the paper terms as “decorative” , and how it tests this claim. Removing one part only shows whether that part was necessary. It does not show whether the model used it. When two parts each suffice for the answer, neither is necessary, so both look decorative, even if the model reasoned from one of them. This is also common among causal inference work. There is also an example from the reported appendix from MedQA, that shows a clinically corroborating evidence concluded as decorative. Other work (eg: ERASER (DeYoung et. al, 2020)) also report metrics like sufficiency, This could be visualised as a 2x2 classifcation (not necessary but sufficient, are redundant rationales; not necessary and not sufficient are truly decorative rationales etc) You could also think of other such conditional ways to prove that a rationale is truly decorative
  • Every model receives the same generation prompt. It asks the model to think step by step and explicitly reference the specific information it uses from the context. Strip-and-regenerate removes the decorative parts and regenerates with the same prompt. If the model already knows the answer, the instruction still obliges it to cite something from the context, so decoration is partly requested. After stripping, the same instruction forces it to cite whatever is left, so re-decoration is the expected outcome of instruction-following.
  • The LLM judge dependency is not validated. Several LLM judges seem to be used for this framework, and a lot of the mapping to sets also rests on these judges. The same LLM is used for judging citations for own inputs, and also writing counterfactuals. This needs strong validation with human LLM alignment scores reported.
  • The analysis overstates the conclusions, and the results could be better explained. Perhaps a more qualitative analysis would also help to understand why these decorative mismatches occur. While the paper acknowledges the limitations and scope of their definition, the analysis makes strong claims like “Uncited context is the most severe form of misalignment” . Suppose two parts both support the answer, cited c₁ and uncited c₂. There are two possible reasons: A (the paper's reading): the model used c₂ and cited c₁ only for show. B: the model actually reasoned from c₁, and c₂ is a backup that happens to support the same answer. In both, removing c₁ leaves the answer unchanged because c₂ still supports it, and removing everything changes it. The two ablations give identical results, so they cannot separate concealment (A) from harmless redundancy (B). The result also claims that “"Most often the model did not need the context at all" using privasis as a control. It is synthetic and recent, so the paper assumes the models cannot have memorized it, and prior knowledge should then be rare. Instead, privasis shows 65.5–76.4% prior knowledge, about the same as the real benchmarks in the appendix.

Minor errors and inconsistencies:

Some proposed content is missing. For example, Sec. 3 says prevalence is first reported over all mentions in Sec. 4.2. Sec. 4.2 and Table 1 report citations only. Confidence intervals are not reported. Some references have incorrect or outdated details. eg:, [76] Sun et al. is cited as the CHI 2026 main proceedings but appeared in the CHI Extended Abstracts. References that are more closely tied to the work (eg: ERASE DeYoung et. al) are not cited

Questions for authors
  • Can the authors add a sufficiency test (retaining only the cited parts) and report what fraction of decorative citations are redundant (sufficient but not necessary) versus truly decorative (neither)?
  • If the other answer-carrying parts are removed first, does ablating the cited part then change the answer? This would help separate redundancy from non-use.
  • The generation prompt explicitly asks the model to reference the context. How sensitive are the decorative rates and the strip-and-regenerate results to this instruction?
Reviewer_QBtfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study “decorative rationalization,” where models cite contextual information as support for an answer that remains unchanged when that information is removed. It defines a removal-based metric and uses repeated interventions with statistical tests to identify citations with high answer-preservation rates. Further interventions distinguish cases associated with prior knowledge, uncited context, and redundant citations. The main evaluation covers approximately 53,000 citations across three datasets and four model configurations, with additional model and synthetic-data experiments. The authors find that removal-insensitive citations are common, reasoning does not consistently reduce them, and stripping such information before regeneration reduces but does not eliminate the phenomenon. The study provides a behavioral audit of citation dependence, although removal insensitivity does not by itself establish that the cited information was unused in the original computation.

Critical strengths
  1. Clear and useful evaluation perspective: The paper clearly distinguishes whether a citation supports an answer from whether the model’s answer changes without it. This provides a useful complement to conventional citation-quality evaluation.

  2. Substantial empirical investigation: The study covers multiple datasets and model configurations, with repeated interventions, statistical testing, and additional joint-removal and regeneration experiments. These analyses provide meaningful evidence that citation removal insensitivity is widespread and not eliminated simply by enabling reasoning or filtering context.

Critical weaknesses
  1. Limited model coverage: GPT-4o-mini is the only proprietary model tested. Stronger, newer models would help show whether the findings hold for current systems.

  2. Unclear practical impact: The paper does not show that decorative citations hurt accuracy or user decisions. Even redundant citations can help readers verify an answer.

  3. Missing human validation: The LLM judges are not checked against human annotations. It is also unclear whether users find the flagged citations misleading or useful.

Questions for authors
  1. Relationship to citation precision: Could you compare decorativeness with ALCE citation precision on the same responses? What additional problems does it detect beyond valid but redundant citations?

  2. Evidence use versus verifiability: Could a model use the evidence initially, then fall back on prior knowledge after removal? How do you distinguish misleading citations from those that still help readers verify the answer?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.