Why Implicit CoT Fails at Complex Logic: Theory, Benchmark, and Solution

Chain-of-Thought (CoT) has unlocked advanced reasoning abilities of Large Language Models (LLMs) with intermediate steps, yet incurs prohibitive computational costs due to generation of extra tokens. Recent studies empirically show that compressing reasoning steps into latent states, or implicit CoT compression, offers a token-efficient alternative. However, the mechanism behind CoT compression remains unclear. In this paper, we provide the first theoretical analysis of the difficulty of learning to internalize intermediate reasoning steps. We introduce *Order-$r$ Interaction* and show that, under uniform attention initialization, the gradient signal identifying relevant inputs decays exponentially with interaction order. Separating this signal from finite-sample fluctuations is ensured by a sample size that grows exponentially with the order. Motivated by the semantic shortcuts that can obscure the evaluation of implicit CoT, we introduce NatBool-DAG, a benchmark of Boolean reasoning expressed in natural language and designed to reduce reliance on such shortcuts. Guided by our theory, we propose ALiCoT (**Al**igned **I**mplicit **CoT**), which aligns latent and explicit reasoning states to enable lower-order computation through latent tokens. Experimental results demonstrate that ALiCoT maintains strong reasoning performance under extreme token compression.