acceptodds
Withdrawn from ICLR 2027

CAIC: A Decision-aware Intervention Framework for Counterfactual Sensitivity in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

20 views

Abstract

Large language models can change consequential decisions after demographic- only edits to otherwise identical inputs. We introduce Counterfactual Activation-based Intervention with Certification (CAIC), a white-box frame- work for reducing decision-level counterfactual sensitivity and bounding what remains. Through a Locate–Forecast–Select–Verify pipeline, CAIC admits low- rank activation interventions only when they satisfy task-specific requirements at the deployed scoring endpoint, and abstains otherwise. We instantiate two op- erators: CAIC-S, a spectral intervention that removes dominant counterfactual directions, and CAIC-T, a task-protected intervention that limits suppression of task-relevant signal. On a controlled task, CAIC-S reduces the mean counterfac- tual gap by 96.6%, with a 95% population upper confidence bound of 0.030. On a semi-synthetic salary-screening task with entangled counterfactual and task sig- nals, CAIC-S reduces final-output raw gaps by 69%–78% across two 7B models, attaining the lowest residual gap among seven matched baselines, though partly through score compression. As an early-exit decision rule at its intervention site, admitted CAIC-T reduces the gap by 32%–74% while better preserving task signal and score structure; however, the mitigation does not persist through downstream computation and CAIC abstains there. Our results underscore that counterfactual intervention should be selected and evaluated at its deployed scoring endpoint.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
bias, LLM, counterfactual, white-box, technical governance, fairness, ethics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HoVnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CAIC, a framework for selecting activation interventions that reduce counterfactual sensitivity of a specified decision score under operational preservation constraints. CAIC-S removes dominant counterfactual displacement directions, while CAIC-T constructs decision aligned directions with penalties for overlap with task relevant contrasts. The framework distinguishes the intervention site from the deployed scoring endpoint, selects configurations on validation data, and reports held out sensitivity bounds.

Experiments include a controlled, bias injected task and semi-synthetic salary screening with two principal 7B models. CAIC-S substantially reduces final output probability gaps, although score compression contributes to these improvements. CAIC-T better preserves several task and score distribution diagnostics at an onsite affine lens endpoint, but its mitigation largely disappears at the native final output.

Critical strengths
  1. The distinction between intervention location and deployed scoring endpoint is useful and supported by informative negative results. Table 1 shows that CAIC-T’s onsite improvements do not survive downstream computation. The paper explicitly reports this limitation and abstains at the final-output endpoint rather than presenting intermediate improvements as successful mitigation of the original model.

  2. The empirical analysis goes beyond raw counterfactual gaps. It includes matched constraint comparisons with seven white box baselines, neutral CDF quantile gaps, score-dispersion retention, threshold changes, and held out uncertainty estimates. Table 7 shows that CAIC-S’s advantage is not entirely explained by score rescaling. The separation of fitting, selection, and held out evaluation is also a methodological strength.

Critical weaknesses
  1. Evidence for practically useful mitigation under adequate preservation remains limited. On the admitted-seed subsets in Table 1, CAIC-S retains only 16.8%–49.4% of neutral-score standard deviation and removes 42.4%–72.0% of the measured task signal. Its admission criteria are primarily ranking-based and therefore tolerate substantial changes in score scale and fixed-threshold decisions. CAIC-T improves this tradeoff, but only through an alternative early-exit affine scorer: final-output reductions are just 0.2%–1.9%, with no final-output configuration admitted in 20 conditions. Onsite admission also varies substantially, including only 1/5 seeds for Mistral race. These findings are transparently reported, but leave the demonstrated deployment benefit narrower than the framework’s motivation.

  2. The incremental value of the certification component is insufficiently established. Theorem 1 follows directly from its Lipschitz assumption, the directional bound uses the standard sigmoid Lipschitz constant, and the population bound is an existing empirical-Bernstein result. In the evaluated finite-family setting, computing the directional certificate uses the same counterfactual representations from which the exact score gaps can already be obtained. The paper distinguishes mean and maximum aggregation correctly, but does not convincingly demonstrate when the directional certificate enables a useful guarantee or operational decision beyond direct held-out gap estimation with a population UCB. I view this primarily as a limitation of contribution and demonstrated usefulness, rather than an identified mathematical error.

Questions for authors
  1. Within the existing candidate sweeps, what happens if CAIC-S and the strongest baselines are selected under common score-dispersion and fixed-threshold preservation requirements, in addition to utility and specificity? Does CAIC-S retain an advantage in quantile-gap reduction at comparable preservation levels?
  2. At the same frozen affine-lens endpoint, how does CAIC-T compare with CAIC-S and representative erasure baselines under identical admission requirements? This would help establish how much of the favorable tradeoff comes from the task-protected operator rather than the change of scoring endpoint.
  3. Can the authors identify a concrete case in the reported experiments where the directional certificate provides actionable information beyond the exact finite-family gap and its empirical-Bernstein UCB? Does it provide coverage beyond the explicitly evaluated counterfactuals under assumptions actually verified in these experiments?
Reviewer_6DceOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines unfair decisions in large language models (LLMs). A decision can change when only a demographic detail, such as a name or gender, changes. The paper proposes CAIC, a post-training method. CAIC compares the hidden representations of matched input pairs. It identifies the low-rank directions that contain the demographic effect. Then, it reduces this effect at a specified layer. CAIC applies the change only if task performance stays acceptable. If task performance decreases, CAIC does not apply the change. It also sets a limit for the remaining sensitivity.

Critical strengths
  1. The method applies an activation change only when it meets two conditions. The change must keep the required task quality at the final decision point. It must also limit the remaining sensitivity. If the change does not meet these conditions, the method does not apply it. The paper also shows that a correction at an intermediate layer can disappear before the model produces the final output. This result is useful for research on model steering and information removal.
  2. The paper shows that the removal of demographic information from model representations does not always make a specified decision fair. It also shows that this removal is not always necessary for a fair decision. Thus, representation-based bias correction must focus on the specified decision.
Critical weaknesses
  1. The method and its certificate apply only to a one-token “Yes” or “No” decision during the prefill stage. They do not apply to decisions that use chain-of-thought reasoning or generated text. Such decisions are common in practical applications.
  2. The paper does not examine the effects of prompt length or demographic-cue position. These factors can affect the counterfactual difference, the stability of the identified directions, and the effectiveness of the correction.
  3. The paper does not discuss or compare methods that give statistical fairness guarantees. It cites the certification method from Chaudhary et al. (2025), but does not use it as a baseline. Thus, the empirical evaluation primarily compares representation-editing methods and simple prompting methods.
Questions for authors
  1. How does the method perform when the decision uses chain-of-thought reasoning? How do changes in prompt length or demographic-cue position affect its performance? In these conditions, do the learned directions stay stable? Does the certificate remain valid? Is there evidence for these conditions beyond the one-step “Yes” or “No” setting?
  2. How does CAIC compare with methods that give statistical fairness guarantees? How does it compare with LLMCert-B (Chaudhary et al., 2025)? Please compare the concepts and the test results. Also, explain why the paper does not use these methods as baselines.
Reviewer_hVFNOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I read the paper several times and couldn't really understand what it is attempting to do, despite familiarity with the topic.

It seems that the paper had too much AI involvement in writing and editing, and too little human attention and effort it making it understandable and readable by humans.

First, the paper jumps from a discussion of LLMs being sensitive to changes in the input (paragraph one) to a question "What should such an intervention remove?" (first sentence of paragraph 2) without the concept of interventions being mentioned at all in paragraph 1. I am familiar with the topic so I could fill in the gap, but a connective paragraph is clearly missing there, and this is already not a good start for readability.

Moving to the technical content, the paper seems to propose a method for intervention in model activations that erases sensitive properties. I could not understand though what is actually being proposed:

  • what is the purpose of Theorem 1? what question is it answering? how is it being used for informing the method, or being used in other way in the paper?
  • I could understand the mathmetical formulation of the intervention, but why these specific ones? what motivates them and what are they attempting to achieve?
  • I could not understand the different stages of the algorithm. For example, for the "locate" stage, the paper only writes: "Using training data, we identify candidate intervention sites and construct their counterfactual displacement matrix D.". I don't understand what this means. How is training data being used? what are candidate intervention sites? how is the displacement matrix constructed? this seems central to the work, but not discussed at all.

There are (many) other points that are unclear, but I will stop at this point.

Critical strengths

Couldn't identify.

Critical weaknesses

AI written. Unreadable by humans.

Questions for authors

Did you really read this text and found it clear and suitable for human readers?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.