CRISP: Interpretable and Adaptive Chain-of-Thought Compression via Small-Model Collaboration

Chain-of-Thought (CoT) reasoning substantially improves the performance of large language models on complex tasks, but often produces lengthy and redundant intermediate steps. Existing approaches delete historical tokens, terminate reasoning early, or compress context into latent representations, but they may require additional training, discard useful information, or produce model-dependent states that are difficult to interpret and reuse. To address these limitations, we propose CRISP, a fine-grained framework for step-level CoT management. CRISP assigns each newly generated reasoning step one of five actions: KEEP, LIGHT, HEAVY, SKIP, and STOP, enabling it to preserve, compress, remove, or terminate reasoning content dynamically. To select the appropriate action, CRISP introduces a lightweight Multi-Agent Controller (MAC), which applies role-specific prompts to a shared small language model and extracts action-aware semantic representations for lightweight probe-based prediction, without retraining the large reasoning model. The same small model also serves as a step compressor: LIGHT removes superficial redundancy, whereas HEAVY distills future-relevant information into compact natural-language states. Unlike latent representations, these compressed states are interpretable, reusable, and transferable across reasoning models. Experiments across multiple benchmarks and model families show that CRISP achieves a favorable accuracy–efficiency trade-off, demonstrating the effectiveness of semantic step-level management for long-chain reasoning.