acceptodds
Withdrawn from ICLR 2027

Don't Let Reasoning Slip Away: Mechanisms and Defense of Implicit Reasoning Risks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) face a distinct risk during long-chain reasoning, known as implicit reasoning risk, in which unimodal inputs that are benign in isolation combine to produce an unsafe output. We find that the recently emerging Thinking with Images (TwI) paradigm reduces this risk relative to the conventional text-chain-of-thought (Text-CoT) multimodal reasoning paradigm. To identify the cause of this behavior, we conduct an in-depth analysis and identify an important issue: the reasoning path of Text-CoT tends to move from concepts toward hypothetical or euphemistic formulations, which yields unsafe responses. We call this phenomenon Semantic Slippage Escape. TwI anchors the reasoning state to concepts and thereby mitigates this escape. Building on this insight, we propose AnchorGuard, a training-free adaptive defense framework that monitors, during early generation, the alignment between the hidden representation and an escape direction and converts this alignment into an intervention strength, which serves as the interpolation weight between the original representation and its projection onto the concept subspace. Experiments show that AnchorGuard improves the safety rate by 9.7% while preserving task utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multimodal Large Language Model, Implicit Reasoning Risk, Thinking with Images, Safety Defense

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UVA9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies implicit reasoning risks in multimodal models, where individually benign inputs can jointly lead to unsafe responses. The authors observe that Thinking with Images (TwI) is safer than Text-CoT and attribute this advantage to reduced “Semantic Slippage Escape.” They propose AnchorGuard, which adaptively projects hidden states toward a concept subspace during generation. Experiments cover four main safety benchmarks, multiple reasoning configurations, and three backbone models. The reported results show improved safety while largely preserving task utility.

Critical strengths

1.In Table 2, AnchorGuard outperforms the strongest defense baseline in each of the six reasoning configurations, improving aggregate safety rates by 3.9–6.6 percentage points. Task utility remains close to the undefended configurations, and results on additional backbones support its broader applicability.

  1. Random-subspace controls, component suppression, and comparisons involving random images and textual descriptions provide evidence about the relationship between task-relevant information and safety.
Critical weaknesses
  1. The escape subspace is constructed from whether responses depart from concrete facts, but factual departure is not equivalent to hypothetical or euphemistic “semantic slippage.” The experiments support the value of concept-related information for safety, but do not sufficiently distinguish general grounding improvements from the proposed slippage mechanism. Consequently, the explanation for why TwI is safer remains incomplete.

  2. The paper does not clearly explain how the six reasoning methods are implemented on the same backbone or which data split is used for hyperparameter selection in Figure 10. These omissions limit assessment of comparison fairness and reproducibility.

Questions for authors
  1. What are the precise annotation criteria for escape/non-escape responses? What evidence distinguishes semantic slippage from ordinary factual errors, visual recognition errors, or differences in response style?

  2. How are the textual descriptions in Table 12 generated? Can the results distinguish recovering previously missed visual information from suppressing semantic slippage during reasoning? Please also clarify the sample size and uncertainty of these results.

  3. Which checkpoints, prompts, and tool procedures are used for the six reasoning configurations? Which data split is used for hyperparameter selection in Figure 10, and how is it kept separate from final evaluation?

1 comment
Reviewer_UVA9OpenReview ↗

Thank you for the clarifications

Thank you for your response. I still have a few questions. First, the new controls help distinguish semantic escape from other errors. However, they do not separate concept anchoring from general grounding improvements. Please make this distinction clear in the revised paper. Thank you for clarifying Q2. However, please answer the question about sample size and uncertainty clearly and precisely. Finally, the reproducibility details are still too general. These details should be easy to state clearly, but the current response does not do so. So, I believe my current rating appropriately reflects the paper’s contributions.

Reviewer_Y9DyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates implicit safety risks in multimodal reasoning, where individually benign image and text inputs can lead to unsafe outputs when combined. The authors identify Semantic Slippage Escape, a phenomenon in which reasoning drifts away from concrete visual concepts toward hypothetical or euphemistic formulations, and observe that Thinking with Images can mitigate this drift. Based on this analysis, they propose AnchorGuard, a training-free defense that uses calibration data to construct representation subspaces and adaptively steers hidden states toward the concept subspace during early generation. Experiments demonstrate improved safety across multiple benchmarks and reasoning configurations, with largely preserved performance on the evaluated utility tasks.

Critical strengths

The main strengths are:

  1. Meaningful research problem. Maintaining grounding and safety throughout long multimodal reasoning is an important challenge. The paper addresses a relevant failure mode in which benign inputs lead to harmful outputs through the reasoning process, making the problem valuable to the broader multimodal learning community.
  2. Simple and effective method. AnchorGuard combines calibration-based subspace estimation with adaptive hidden-state intervention. The approach is conceptually straightforward, requires no model parameter updates, and provides a concrete way to translate the representation analysis into a practical defense.
  3. Positive empirical results. The method consistently improves aggregate safety rates across the evaluated reasoning configurations and outperforms the compared defenses on the main benchmarks. The intervention and ablation experiments also provide useful evidence supporting the proposed design.
Critical weaknesses

My main weaknesses and concerns are:

  1. Generality across model families remains insufficiently established. The evaluation includes Qwen2.5-VL-7B, Qwen3-VL-8B, and LLaVA-OneVision-7B, but remains limited to a relatively narrow range of model scales and architectures. It is unclear whether the same semantic-slippage mechanism persists in newer natively multimodal models, such as Qwen3.5 and Kimi-K2.5, and whether AnchorGuard remains effective on them. Testing both the mechanism and the defense on a representative model from these families would strengthen the paper. Results on more recent models such as DeepSeek-V4.1-Flash would also be informative, although their inclusion need not be essential given their recency.
  2. The inference overhead is substantial. The reported average latency increases from 3.17 to 4.42 seconds per answer, corresponding to approximately 39% overhead. This increase could affect practical deployment, particularly in latency-sensitive applications. A more detailed analysis of throughput, memory overhead, and latency under different generation lengths would help assess the practical safety–efficiency trade-off.
  3. The robustness of calibration is not sufficiently examined. The subspaces are estimated using 500 MMBench samples. Although the evaluation on separate benchmarks provides evidence of transfer, it remains unclear how sensitive the method is to the size, composition, and domain coverage of this calibration set. In particular, generalization to complex safety scenarios combining unfamiliar domains is not established. Experiments varying calibration size and source, together with evaluation on explicitly held-out domains or domain combinations, would help address this concern.
  4. Preservation of general capabilities is only partially evaluated. The utility evaluation focuses on V*, HR-Bench 4K, and HR-Bench 8K, which primarily assess visual understanding and visually grounded reasoning. These results do not establish that hidden-state intervention preserves general instruction following or more complex reasoning capabilities. Broader evaluation on demanding reasoning and instruction-following tasks is needed to support the claim that AnchorGuard does not impair the model’s reasoning ability.
Questions for authors

Please see weakness above.

Reviewer_UushOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates Semantic Slippage Escape, a phenomenon in which multimodal reasoning drifts from concrete concepts toward hypothetical or euphemistic formulations, potentially producing unsafe responses. The authors find that Thinking with Images delays this drift and propose AnchorGuard, a training-free defense that adaptively projects hidden representations toward a concept subspace. Experiments show improved safety across multiple benchmarks and reasoning methods while largely preserving task utility.

Critical strengths
  1. The paper offers an interesting account of how multimodal reasoning can produce unsafe responses and translates it into a concrete inference-time defense. The consistent safety improvements with small utility changes make the approach practically useful.

  2. The experiments include meaningful controls, such as matched random-subspace interventions and image versus caption reinsertion. These help assess why the intervention works beyond the main performance comparisons.

Critical weaknesses
  1. The interpretation of the subspaces needs stronger validation. It remains unclear whether (G) and (H) specifically capture concrete concepts and semantic escape as claimed. Table 11 also shows a substantial additional safety gain from suppressing the residual component alongside (H), leaving part of the method’s benefit unexplained.

  2. The connection to TwI remains incomplete. AnchorGuard works independently of image revisiting, while caption reinsertion recovers much of TwI’s benefit. These findings support the value of refreshing task-relevant information, but do not fully establish the proposed explanation of TwI’s behavior.

Questions for authors
  1. Could you further validate the interpretation of (G) and (H), and explain the additional benefit of suppressing the residual component?

  2. To what extent does the proposed mechanism explain TwI’s safety improvement beyond refreshing task-relevant information?

  3. Could you provide representative escape/non-escape response pairs from the MMBench calibration set to illustrate the differences used to construct (H)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.