acceptodds
Withdrawn from ICLR 2027

CICA-TRACE: Aligning Refusal Decisions with Decisive Safety Evidence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A model can produce the correct refusal while relying on spurious cues instead of the safety evidence that actually warrants it. To address this mismatch, we propose CICA-TRACE, a framework for training refusal decisions to follow decisive safety evidence. CICA-TRACE represents the refuse-or-follow decision with an explicit pre-generation route score and couples matched counterfactual prompts so that changes in safety-critical conditions induce the expected route shift. It further compares interventions on decisive evidence and matched comparison text, encouraging the route to depend more strongly on the former while preserving following behavior on benign requests. We evaluate this behavior with Route Faithfulness (RF) and Attribution Gap (AG), which measure correct route movement and preferential reliance on decisive evidence, respectively. Across five open-weight LLMs, CICA-TRACE consistently improves RF and AG while substantially reducing harmful responses. Moreover, on five LLMs, CICA-TRACE reduces the mean harmful rate from 11.25% to 2.46%, while keeping general utility close to the base models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model Safety, Safety Alignment, Refusal Alignment, Counterfactual Attribution, Jailbreak Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QURDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to address the problem of "shortcut refusal", where the LLMs might wrongly refuse a request just based on the superficial feature of the request (e.g., decision is based on just certain topic words). To this end, the paper provides TRACE, a set of matched prompts that differ in one safety condition, and measure the refusal decision before answer generation. Using the two defined metrics, Route Faithfulness (RF) and Attribution Gap (AG), we can identify the decision safety evidence from the input. Finally, the authors propose CICA-TRACE, a method that aligns refusal decisions with decisive safety evidence.

Critical strengths
  • The focused problem is well-motivated.
  • The authors provide a dataset that can be beneficiary to the community.
Critical weaknesses
  • Not sure whether the counterfactual sample generated by the method is reliable, or there are also other errors introduced. It also remains unknown how do the authors ensure the edited statements represent meaningful changes rather than some artificial errors.
  • When constructing TRACE, only individual evidence intervention is applied. However, real-world safety decisions oftend depend on the interactions among multiple evidence.
Questions for authors
  • How are the counterfactual intent and policy edits constructed and validated? Any human annotators involved? This is pretty important for evaluuating the quality of the constructed models.
Reviewer_kycQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work studies how and why a model might refuse a request. Most safety evaluations focus on the output outcome (i.e. refusal or compliance), rather than checking if the model refused for the correct reason (i.e. the user has or does not have the correct authorization for a query). In contrast, this work specifically studies what parts of a prompt results in the refusal by building a dataset of almost identical input pairs that differ by a specific detail which makes it decidedly unsafe. Then, they measure whether this change appropriately modifies the generation outcome (route faithfulness), and if the change in response is due to the correct portion of the prompt rather than blanking out incidental text (attribution gap). Initially, they find that all five base models evaluated correctly shift the response between refusal and compliance for a given change (i.e. positive route faithfulness), but are often swayed more by incidental text (poor attribution gap). The authors propose to remedy this with their CICA-Trace framework, which trains the model to use the correct evidence within a prompt to decide whether to refuse or comply.

Critical strengths
  • Tackles an interesting, difficult, and important problem of correctly attributing refusal decisions to specific/relevant bits of information in the input prompts. This has been a recurring problem in safety training and approaches like this should in theory help with disambiguating edge case/dual use inputs better.

  • The experimental work is quite thorough in terms of models and evaluations.

Critical weaknesses
  • The writing was not easy to follow. Lots of sentences have ambiguous jargon that is not properly defined. Many segments read like LLM generated content which takes many liberties on what is assumed to be known by the reader. Many terms are used before defining them, or without defining them at all (eg. “route,” “route verbalizer,” “stage grounding,” “route equivariance,” “comparison dominance”). It was difficult to really understand what were the main takeaways from your work other than your training framework improved the metrics (RF, AG) that you defined. I’d recommend significant effort on a major re-write of the work, not just polishing it.

  • I’m surprised this approach increases over-refusal in XSTest and OR-Bench. Since this was one of the motivators of your work, I would have liked to see a better understanding for why this is the case.

  • Training includes an auxiliary loss on 192 trace-free prompts (”anchors”?) paired with a fixed refusal (Appendix A.2), which is only briefly mentioned in the main text and (I believe) is retained in every ablation (Appendix A.6). This term may account for much of the reduction in harmful responses, independent of the proposed route and evidence objectives, as well as contributing to the increase in over-refusal. Could you report results without this term, and for a model trained with this term alone?

  • Weak/missing adversarial evaluations; the algorithmic attacks used (GCG (25 steps, 32 candidates), Best-of-N (64)) has a way smaller compute budget than what is recommended (e.g. ~500-1k GCG steps + 512 candidates, N=1k+). Also, there are no other common attacks such as PAIR, nor adaptive attacks that exploit the behaviour you train for. Since the model is trained to follow the intent and policy statements in its reasoning trace, I’d expect the most effective attack to be simply writing those statements into the prompt, e.g. claiming the intent is benign and no policy applies. Could this kind of injection flip the model toward complying on harmful requests? This could then serve as an objective to optimize for with the other jailbreak methods.

Questions for authors
  • The AG score seems like it could be negative by design without training if the model’s ‘routing’ barely responds to anything. The decisive term keeps its sign, so averages over this could be zero if they’re just noise whereas the comparison term is absolute, so you’d always be subtracting a positive value. Could you compute AG with a span that shouldn’t matter standing in for the decisive one, to show what AG looks like when there’s no real evidence use?

  • My understanding is that you train the model with a safety reasoning trace (fig. 8 + stage grounding); does the trained model generate this reasoning trace at inference? If not, what is the stage training contributing? This is quite quite unclear.

Reviewer_fGwhOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that refusals could be caused by safety relevant evidence or by spurious cues. They score the refusal vs compliance gap by looking at logprobs, and use this to build two metrics, RF and AG, that measure the impact of safety evidence vs spurious cues. They evaluate these metrics on the TRACE dataset they generate starting from Armor reasoning traces. They propose different training losses based on RF and AG and train on the train split of the generated dataset. Results improve on RF and AG while reducing harmful responses on regular safety benchmarks.

Critical strengths
  • The paper identifies a real problem in refusal behaviour of llms, and presents it very clearly
  • RF and AG are properly define to quantify this problem
Critical weaknesses

The main text never mentions how TRACE data is generated. A reader may conclude that an average dataset entry looks like the one shown in Figure 1. But in practice, if my understanding is correct, data looks quite different from it. A dataset entry is obtained by editing a fixed CoT format from Armor, which could introduce several problems. These problems could be 1. that during training the model just learns to only consider the "policy" entry and 2. that evaluations using the dataset are relying on prefilling off policy reasoning traces. Additionally, training on this very specific format and also evaluating on it, makes it so the gains in RF and AG could appear just by construction

Questions for authors

Could you clarify the dataset creation process and address how well the method's RF and AG generalise on prompts without the Armor format?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.