acceptodds
Under review as a conference paper at ICLR 2027

When Representations Collapse in Logical Reasoning: Diagnosing and Mitigating Cascade Failures in LLMs

Abstract

Large language models (LLMs) achieve strong performance on logical reasoning benchmarks, yet remain brittle as proofs become deeper, irrelevant premises accumulate, or evaluation conditions shift. We investigate this brittleness at the level of internal representations. To this end, we introduce Inferential Information Flow (IIF), a reasoning-aware diagnostic framework that measures, layer by layer, whether a model preserves selectivity toward proof-relevant evidence over distractors. Across three model families and three logical reasoning benchmarks, IIF reveals a broad but heterogeneous failure pattern: proof-selective structure often strengthens at intermediate depth and subsequently degrades, while some settings instead fail to form strong selectivity at the query interface. We term the former cascade IIF collapse. Causal interventions further show that some degraded value-channel representations can become harmful to downstream reasoning rather than merely uninformative. Motivated by this diagnosis, we introduce FRGI (Fact-Rule Grouped Intervention), a training-free, proof-free inference-time method that uses query-conditioned saliency to select candidate fact and rule spans and an Unlabeled Collapse Proxy (UCP) to locate intervention layers. FRGI applies localized fact-rule and value-channel restoration without retraining, proof annotations, or prompt modification. Across ProverQA, PrOntoQA, and ProofWriter, FRGI consistently improves accuracy over representative prompting, self-consistency, symbolic-reasoning, and activation-intervention baselines. These results show that representation-level failures in logical reasoning can be diagnosed and partially repaired through targeted inference-time intervention.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.