acceptodds
Under review as a conference paper at ICLR 2027

Nuisance-Induced Confounding in LLM Output Sensitivity Analysis: A Semantic Normalization Framework

Abstract

Distribution-based sensitivity analysis aims to quantify how large language model (LLM) outputs change under input perturbations, but raw output distributions can conflate semantic changes with nuisance variation from stochastic generation, surface form, language, and representation choice. We formalize this nuisance-confounding problem by modeling LLM outputs as mixtures of meaning and nuisance factors, showing that raw-output distributional sensitivity can be nonzero even when the underlying meaning is unchanged. Motivated by a nuisance-invariance criterion for meaning-centered sensitivity analysis, we formulate a semantic normalization framework that maps raw outputs into meaning-focused representations before distributional comparison. From a conditional information bottleneck perspective, this transformation is interpreted as approximately preserving meaning while suppressing nuisance-dependent variation. Experiments on controlled perturbation analyses show that meaning-preserving nuisance changes induce nonzero raw-output sensitivity, whereas summary-based normalization reduces this nuisance-induced sensitivity by 53.6% on average while retaining task-relevant content. In multilingual patient-data experiments with controlled semantic changes, summary-based normalization yields more comparable sensitivity estimates across languages. These results characterize nuisance confounding as a measurement problem and support summary-based semantic normalization as a practical mechanism for mitigating its effects.

Primary area:
interpretability and explainable AI
Keywords:
LLM sensitivity analysis, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2EycOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies nuisance-induced confounding in distribution-based sensitivity analysis of LLM outputs. The authors argue that raw output distributions may conflate semantic changes with nuisance variation arising from factors such as stochastic generation, surface form, language, and representation choice. They formalize this phenomenon through a latent-variable model in which outputs depend on both semantic meaning and nuisance factors, show that raw-output sensitivity can be non-zero even when meaning is unchanged, and derive a sufficient condition for nuisance-invariant sensitivity analysis. Motivated by this analysis, they propose a semantic-normalization framework based on summarizing outputs prior to computing distributional sensitivity. Experiments on nuisance perturbations, controlled semantic perturbations, and multilingual settings suggest that summary-based normalization reduces sensitivity to nuisance variation while maintaining sensitivity to semantically meaningful changes.

Critical strengths

(1) The paper provides a clear measurement-theoretic formalization of nuisance effects in distribution-based sensitivity analysis. The theoretical development seems coherent and motivates a concrete invariance criterion for representations used in sensitivity estimation. (2) The paper does more than present a theoretical argument; it also proposes a simple normalization procedure and evaluates it across several controlled settings, including multilingual experiments. The empirical results are broadly consistent with the motivating intuition.

Critical weaknesses

(1) I think the paper's strongest contribution is narrower than the paper currently suggests. The theoretical framework and empirical results support the claim that distribution-based sensitivity metrics measure a mixture of semantic and non-semantic variation, and that a normalization step may be useful when the goal is specifically to quantify semantic change. However, I am less convinced that the underlying observation that nuisance variation can affect measured sensitivity is itself novel. Recent work already studies distribution-based auditing [1,2], semantics-preserving perturbations [3], semantic consistency and robustness [4,5], and semantic clustering/equivalence of outputs [6,7,8]. In that context, a theorem showing that nuisance variation can affect output distributions when meaning is fixed does not by itself feel like a major discovery.

(2) Because of this, I think the related-work discussion and positioning need to be strengthened substantially. In particular, I would like to see a clearer discussion of how the proposed framework differs from existing work on semantic consistency, semantics-preserving perturbations, semantic clustering of outputs, and distribution-based sensitivity analysis [1-8]. More generally, if the key claim is that raw output-distribution comparisons do not identify semantic sensitivity without additional assumptions, then it is not clear to me what gap remains despite these increasingly semantics-aware approaches. I also found it difficult to interpret the practical significance of some reported improvements, as several reductions are presented as large percentage gains while corresponding to relatively small absolute changes in the sensitivity metric.

Refs: [1] Rauba et al. (2024), Distribution-based Sensitivity Analysis for Large Language Models. [2] Rauba et al. (2025), Statistical Hypothesis Testing for Auditing Robustness in Language Models. [3] Romanou et al. (2026), Brittlebench: Quantifying LLM Robustness via Prompt Sensitivity. [4] Hejabi et al. (2026), Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs. [5] Pinneri et al (2025), Guarding the Meaning: Self-Supervised Training for Semantic Robustness in Guard Models. [6] Nguyen et al. (2025), Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity. [7] Kunitomo-Jacquin et al. (2026), Evidential Semantic Entropy for LLM Uncertainty Quantification. [8] Karimi et al. (2026), LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy.

Questions for authors

Following the weaknesses discussed above, recent work already studies distribution-based auditing, semantics-preserving perturbations, semantic consistency/robustness, and semantic clustering of outputs [1-8]. Could the authors clarify what gap remains despite the number of semantics-aware contemporary approaches , and why the nuisance-confounding problem identified in this paper should be viewed as a distinct contribution rather than a formalization of limitations that are already recognized by this literature?

1 comment

We thank the reviewer for the thoughtful feedback. We clarify our distinction from prior work, emphasizing the theoretical formulation of nuisance-induced confounding and its empirical implications.


Distinction from Prior Work and Theoretical Contribution

Our contribution is not the first observation that surface-form variation affects LLM outputs. Rather, we formalize nuisance-induced confounding as a structural measurement problem in distribution-based sensitivity analysis, establish a criterion for nuisance-invariant representations, and connect this criterion to a practical normalization mechanism.

Distribution-based sensitivity analysis [1–2] provides methods for quantifying changes in LLM outputs. In particular, [2] constructs distributions in a semantic-similarity space and evaluates perturbation discrimination using FPR, TPR, and AUC. However, semantic-similarity-based comparison does not by itself ensure that measured sensitivity isolates task-relevant semantic changes. Nuisance-dependent output variation can induce nonzero distributional sensitivity even at the population level when underlying meaning remains unchanged.

Theorem 1 formalizes this limitation through a meaning–nuisance mixture model, showing that shifts in nuisance mixing weights can alter raw-output distributions even when meaning is fixed. Proposition 1 then provides a sufficient condition for avoiding such confounding: conditional independence of the representation from nuisance given meaning. Eq. (7)'s conditional information bottleneck (CIB) objective expresses the corresponding design goals of preserving meaning-related information and suppressing nuisance dependence.

Semantics-preserving perturbation and robustness studies [3–5] examine model behavior under input variations. [3] BrittleBench decomposes performance variability, while [4–5] study prediction consistency and semantic robustness. These studies also address nuisance-related variability; our complementary focus is how such variation confounds the interpretation of distribution-based output sensitivity.

Semantic uncertainty studies [6–8] use semantic similarity or clustering to estimate uncertainty over generated meanings. While these methods account for linguistic variation, our focus is on whether distributional sensitivity across input conditions can be attributed to task-relevant meaning changes rather than nuisance variation.

Motivated by this formulation, we introduce summary-based semantic normalization as a practical approximation to nuisance-invariant representations. Holding generated responses and sensitivity computation fixed, we compare Raw, Prefix Truncation, Format-Only, Extractive Length-Matched, and Summary to assess nuisance suppression and semantic discrimination (Table 4). Table 5 further examines nuisance-information suppression and task-relevant semantic information retention.

Absolute Sensitivity Changes and Discrimination Performance

We agree that relative percentage reductions alone do not establish practical significance. Table 2 reports the following absolute changes in the controlled refund-policy task:

MetricRawSummaryChange
Mean nuisance sensitivity (ω)0.17510.0524−0.1227
Mean semantic sensitivity (ω)0.19240.3558+0.1634
Discrimination AUC0.45310.8854+0.4323

Summary reduced nuisance sensitivity while increasing semantic sensitivity, rather than uniformly suppressing both. The paired Summary–Raw ΔAUC was +0.4323 [0.2943, 0.4818] based on 5,000 bootstrap resamples.

We also revalidated the length-matched extractive control, achieving exact Gemma token-length matching for 320/320 response pairs. Under the same local encoder configuration, Extractive achieved an AUC of 0.7135, versus 0.8854 for Summary. The paired difference remained +0.1719 [0.0625, 0.3282], indicating an improvement beyond length reduction alone.

Additionally, Appendix D.3.4 reports an average nuisance-sensitivity reduction of 0.0343 [0.0257, 0.0445] after adjusting for same-input generation variability in low-Raw-sensitivity settings, with reductions in 49/64 task–condition–encoder combinations.

These results demonstrate that the contribution is not limited to relative reductions in sensitivity scores: summary-based normalization improves discrimination between the evaluated semantic and nuisance perturbations, providing empirical support for the measurement framework developed in our theoretical analysis.

Reviewer_BUMaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The main problem is interesting. Comparing the embedding measures, style and substance both, while we want only substance to be measured. We don't want changes in tone, length, format, or language to matter as much as the content.

The authors try to introduce a theory that splits the output into meaning and nuisance. They hypothesize that a nuisance shift can cause a non-zero sensitivity score. Thus, they claim that a representation that depends only on meaning would avoid this problem. Summarization is offered as this alternative.

Thus, the overall method is to summarize every output and then compute the sensitivity.

Critical strengths

Interesting problem statement.

Critical weaknesses

The evidence in the results section is weak. Finite-sample noise is likely a major contributor to the results and is not quantified. The summarizer is also a confounder. The stated theory is a tautology. The proposed method and theory are not in line with each other (or rather, there are many other possible explanations and many other methods for the "theory"). Results seem to depend on the encoder (perhaps it should), and the small-scale experiments make it difficult to appreciate the core contribution. The required changes are too much for a rebuttal to correct.

Questions for authors

Nothing specific that could materially affect my assessment. The required changes are too much for a rebuttal to correct.

2 comments

We thank the reviewer for the feedback. We address the concerns regarding finite-sample reliability, the theoretical motivation, the effects of summarization, and encoder dependence.


Theoretical Contribution and Empirical Evidence

Our theory addresses nuisance-induced confounding as a structural measurement problem, distinct from finite-sample estimation error. Even with accurately estimated output distributions, changes in nuisance factors can produce nonzero distributional sensitivity when the underlying meaning is unchanged.

The role of our theoretical formulation is to make this measurement limitation explicit, establish a criterion for appropriate output representations, and guide experimental design and interpretation. Theorem 1 formalizes how shifts in nuisance mixing weights can change raw-output distributions without semantic changes. Proposition 1 provides a sufficient condition for nuisance invariance through conditional independence of the representation from nuisance given meaning. Eq. (7)'s conditional information bottleneck (CIB) objective expresses the corresponding goals of preserving meaning while suppressing nuisance dependence.

Summary-based normalization is a practical mechanism motivated by this criterion, rather than a method whose invariance is theoretically guaranteed. We therefore evaluate whether it exhibits the intended properties empirically. Table 5 measures linear-probe recoverability from frozen embeddings, testing nuisance labels on held-out semantic states and semantic labels on held-out nuisance states.

Probe TargetRaw Macro-F1Summary Macro-F1
Tone0.9100.385
Format0.9860.475
Length0.9290.557
Refund decision0.8830.900
Fine-grained policy state0.9020.899

Nuisance-label recoverability decreased substantially, whereas refund-decision and fine-grained policy-state recoverability was maintained. Together with the improved semantic-vs-nuisance discrimination in Table 4, these findings provide empirical evidence consistent with the representation-level goals identified by our theory: suppressing nuisance-related variation while preserving task-relevant semantic information.

Finite-Sample Reliability and Length-Matched Controls

We performed 5,000 paired bootstrap resamples over generated responses for a fixed set of eight nuisance and eight semantic refund perturbations, sharing response indices across representations and encoders. JSD-based sensitivity and AUC were recomputed in each replicate.

SettingComparisonΔAUC95% Bootstrap CI
n = 10Summary − Raw+0.4323[0.2943, 0.4818]
n = 50Summary − Raw+0.4219[0.3698, 0.4583]
MatchedSummary − Extractive+0.1719[0.0625, 0.3282]

ΔAUC values are averaged across three encoders.

The improvement persisted with a larger response pool (Table 6). We also corrected the extractive control and verified exact Gemma token-length matching for all 320/320 response pairs. Under the same local encoder configuration, Extractive achieved an AUC of 0.7135, versus 0.8854 for Summary.

These results show that the discrimination advantage persists beyond response-sampling variability and length reduction under the evaluated conditions.

Robustness Across Embedding Encoders

Embedding-based sensitivity estimates can vary in absolute magnitude and condition-level behavior across encoders. An important question is therefore whether the improvements associated with summary-based normalization persist across different embedding spaces.

In the submitted Table 7, we evaluated seven nuisance perturbations across four tasks using six embedding encoders. All six encoders showed a reduction in mean nuisance sensitivity after summarization, and five of the six showed reductions in 17–25 of the 28 task–condition combinations.

These results indicate that although effect magnitudes and condition-level behavior vary across encoders, the reduction in mean nuisance sensitivity is not confined to a single embedding encoder.

Thanks for the response. The clarifications don't clarify the critical issues. The utility of the stated theory is still unclear and the reliability of the experiments are still an issue. I maintain my original rating.

Reviewer_EdgEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This kind of distribution-based sensitivity analysis samples model outputs before and after a perturbation, embeds them, and then compares the distributions. The authors argue that this metric conflates two things: whether the meaning itself changed, and whether only the nuisance used to express the meaning changed. They model the output as generated from a meaning variable and a nuisance variable; Theorem 1 states that if the two nuisance components differ and their mixing weights differ between the two inputs, then the output distributions differ, so even with unchanged meaning the measured sensitivity is nonzero. Proposition 1 gives a sufficient condition: if the representation is conditionally independent of nuisance given meaning, then its distribution is invariant to nuisance shifts. The ideal summary is then viewed as the optimal solution of a conditional information bottleneck (Section 2.3). The concrete recommendation is: summarize each response first, then run the original embedding and divergence pipeline. The evidence comes from four customer-service tasks, seven meaning-preserving perturbations, and a refund task with eight nuisances and eight semantic perturbations, plus multilingual patient data.

Critical strengths
  1. The controlled experiments are more thorough than most work in this area, and the Table 4 ladder is the reason I take the result seriously: Raw 0.45, Format-Only 0.51, Extractive 0.68, Summary 0.89, with prefix truncation below Raw. That pattern rules out simple formatting cleanup and length reduction as the whole story. The probes point the same way: the nuisance probes drop a lot, while the semantic probes barely move.
  2. Counterexamples are reported too — E5 is much weaker than the other encoders, and the appendix tables have cells where summarization increases sensitivity. I appreciate that.
Critical weaknesses
  1. The theory is thinner than the framing suggests. Theorem 1 is a short algebraic argument, and Proposition 1 is essentially the definition of conditional independence. The CIB objective is called a conceptual principle and is never optimized or bounded, and the regularity condition on p.6 is asserted but not stated. I also couldn't reconstruct the step after Eq. (13): why does a small I(Psi(Y);Y|S) imply a small Delta? Maybe this is standard, but as written I can't check it. The paper also doesn't give a reason to prefer 50-150 tokens or one summarizer, and the failure discussion is qualitative. That's not fatal for a measurement paper, but the theory isn't doing much work.
  2. The main empirical weakness is that the headline AUC rests on a small controlled set — 8+8 perturbations, ten samples, one generator — and Table 2 has no CI. The appendix does report subsample variability and other generators, so this is more of a presentation issue than a hidden problem. I'd still like a CI on the main table. The gap after the extractive control is also not fully explained. One alternative is that summarization just contracts the embedding cloud; Table 4 argues against that, since semantic sensitivity goes up rather than down, but a direct spread analysis would settle it.
Questions for authors
  1. The extractive control is length-matched, so the remaining gap isn't length. Is it abstraction, or is MMR keeping content that the summarizer drops?
  2. Semantic sensitivity actually rises after normalization. Is that pre-existing variation in the raw outputs, or is summarization amplifying small policy differences?
  3. A bootstrap interval for the summary-vs-extractive gap would be useful; the permutation test is already there.
  4. Breaking semantic sensitivity down by perturbation type would separate the two readings in item 2.
3 comments

We thank the reviewer for the constructive feedback.


Theoretical Contribution and Scope

Our theory serves a measurement-focused role: it formalizes how nuisance variation can confound the interpretation of distributional sensitivity, establishes a sufficient condition for nuisance-invariant representations, and provides a principled basis for designing and interpreting controlled evaluations.

Specifically, Theorem 1 shows that shifts in nuisance mixing weights can alter raw-output distributions even when meaning is fixed, provided the nuisance-conditional components differ. This reflects structural measurement confounding, not merely finite-sample error. Proposition 1 provides a sufficient condition for invariance: a representation conditionally independent of nuisance given meaning remains invariant to shifts in nuisance mixing. Eq. (7)'s conditional information bottleneck (CIB) objective translates these principles into the complementary goals of meaning preservation and nuisance suppression; it is not directly optimized.

This formulation guides our evaluation: summary-based normalization should suppress nuisance sensitivity while preserving task-relevant semantic sensitivity. We examine these goals through controlled representation comparisons (Table 4) and nuisance/semantic information probes (Table 5).

Regarding Eq. (13), it is a variance decomposition relating an idealized conditional-expectation reference to a practical summary-based score through approximation error Δ and covariance. It does not establish that the practical score is a Rao–Blackwell estimator. Likewise, small I(Ψ(Y);Y∣S)I(\Psi(Y);Y\mid S) alone does not imply a quantitative bound on Δ without additional assumptions. We will clarify this distinction.

AUC Confidence Intervals and Exact Length Matching

We performed 5,000 paired bootstrap resamples of generated responses for fixed sets of eight nuisance and eight semantic refund perturbations, sharing indices across representations and three encoders. We recomputed JSD and encoder-wise AUC in every replicate and report 95% percentile intervals for the three-encoder average.

Table 2 (results with additional bootstrap confidence intervals).

RepresentationNuisance ωSemantic ωFPRTPRAUC95% CI
Raw0.17510.19240.670.790.4531[0.4115, 0.5678]
Summary0.05240.35580.420.920.8854[0.8125, 0.9375]

The paired Summary–Raw ΔAUC was +0.4323 [0.2943, 0.4818]; at n=50, it was +0.4219 [0.3698, 0.4583].

All 320 Extractive–Summary pairs matched exactly in Gemma token length. Extractive and Summary AUCs were 0.7135 and 0.8854, respectively (paired ΔAUC +0.1719 [0.0625, 0.3282]).

Task-Relevant Content Retention

We audited 80 base/perturbed responses from four semantic conditions selected at both extremes of sensitivity change.

Retained from RawExtractiveSummary
Refund decision31/80 (38.8%)72/80 (90.0%)
Core policy rationale13/80 (16.3%)37/80 (46.3%)

Unlike MMR-based extraction, summarization can consolidate decisions and rationales distributed across a response. Its higher retention rates under identical token budgets suggest that content selection and consolidation, beyond length reduction, contribute to the discrimination gap.

Embedding Spread and Semantic Perturbation Breakdown

Mean cosine distances across three encoders and eight conditions per perturbation type.

Perturbation / DistanceRawExtractiveSummary
Nuisance / within-condition pairwise0.082850.285320.13694
Semantic / within-condition pairwise0.076130.283200.17157
Nuisance / base–perturbed centroid0.066780.106300.04353
Semantic / base–perturbed centroid0.058550.131570.16984

Summary increased within-condition spread versus Raw in all 16 conditions, and centroid distances shifted in opposite directions by perturbation type, arguing against uniform contraction.

Semantic sensitivity increased in seven of eight perturbations.

Semantic perturbationRaw ωSummary ω
Exchange-only0.19350.5623
Final sale0.37860.6573
Changed mind0.21240.3721
No receipt0.07250.2193
Seven-day window0.12370.2667
Opened item0.47630.6123
45-day return0.04510.1358
Customer-caused damage0.03740.0206

The increases span multiple policy changes rather than a single perturbation type. Together with the decision- and rationale-retention results and the selective changes in centroid distances, this pattern is consistent with summarization making existing policy distinctions more salient. We interpret the sensitivity increases alongside content retention rather than as evidence of semantic fidelity on their own.

Data Provenance

All patient profiles used in our experiments were synthetically generated and did not represent real patients.

Reviewer_EdgEOpenReview ↗

Response to the Authors

Thank you to the authors for the detailed response, which addresses some of my concerns.

Thank you for considering our detailed response. We appreciate your thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.