- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Modular Chain-of-Cognition to reorganize computation within frozen language models using function-guided feedback. It derives a global ordering of cognitive functions from annotated reasoning decompositions and associates each function with selected attention heads. Mismatches between this ordering and the heads' layer locations define feedback connections that expose earlier layers to deeper representations. The resulting routing plan aggregates selected head outputs and recomputes downstream layers during the prefill pass. Experiments on language and vision-language reasoning benchmarks show average accuracy gains over standard inference, with smaller improvements in the multimodal setting and additional prefill cost.
Critical strengths
- The idea is interesting. The method turns annotated functional structure into an explicit inference procedure that can be applied without updating the backbone or training a routing policy.
- The paper is well-written and easy to follow.
Critical weaknesses
- The routing criterion combines correlational head attribution with average first-occurrence positions in human decompositions. Neither establishes that a target component requires the source component's output, or that their layer ordering constitutes a computational error. Moreover, the injected signal enters the shared residual stream and affects all subsequent heads and MLPs. The accuracy gains therefore provide limited evidence for the central interpretation that identifiable cognitive modules have been composed in a more appropriate order, rather than that selected activation feedback improves computation more generally.
- Table 6 does not isolate the effect of cognitive ordering as claimed. Permuting the function labels changes the selected source-target connections and can change the number of hops, as the two-hop and three-hop controls demonstrate. These interventions differ in the information injected and the computation performed, in addition to their ordering. Keeping the head sets fixed is insufficient to resolve this confound. A control that matches the routing graph and recomputation budget while changing the execution order would more directly test the proposed mechanism.
Questions for authors
- Which routing plan was used for the reported Llama-3.1-8B results? Eq. (8) starts with layer 14 to Input(13), whereas Appendix A.1 describes layer 25 to layer 12 for Retrieval. Could the authors reconcile the source layers and specify the injection sites with a consistent indexing convention?
- How are the nonlinear probes fitted and the per-function top-k values selected? Were the probing examples, ordering annotations, and injection-strength selection separated from benchmark evaluation examples, given that CogQA is assembled from reasoning benchmarks?
3 comments
Next
Q. Which routing plan was used for the reported Llama-3.1-8B results?
We thank the reviewer for catching the inconsistencies. The routing plan used in all experiments on Llama-3.1-8B is:
14 → Input(12), 25 → Input(23), {24, 25} → Input(13)
The experiments were not affected; only the descriptions were inconsistent. We will correct both in the revision.
Q. How are the nonlinear probes fitted and the per-function top-k values selected? Were the probing examples, ordering annotations, and injection-strength selection separated from benchmark evaluation examples, given that CogQA is assembled from reasoning benchmarks?
Nonlinear Probing and Head Selection. Following [1], we use nonlinear probing to identify attention heads associated with each cognitive function. Head importance is quantified through probe-based attribution, and the top-(k) heads are selected according to their importance scores. The selected heads are localized once offline for each backbone and subsequently reused across downstream reasoning tasks without additional probing or parameter updates. We will provide further details on probe training and the per-function selection of k to improve reproducibility.
""Evaluation Data.** We matched every question in CogQA (Ma et al., 2026) and CogVision (Jiang et al., 2025) against all evaluation sets using 8-gram text overlap and, for CogVision, perceptual image hashing.
CogQA: no overlap. Some CogQA questions match the GSM8K and AQuA training sets (114 and 105, respectively), but none match the GSM8K or AQuA test sets used for evaluation, and none overlap with LogiQA, MMLU, WinoGrande, PIQA or RTE.
CogVision: partial overlap. We found 173 CogVision items that also appear in our MathVista evaluation set and 116 that appear in our MMMU evaluation set, and none from ScienceQA or GQA. After excluding the overlapping questions, MoCoC's macro-average gain over the baseline remains positive on all four VLMs:
| Model | Full sets | De-duplicated |
|---|---|---|
| Qwen2.5-VL-7B | +1.0 | +0.6 |
| Qwen2.5-VL-3B | +0.4 | +0.2 |
| Gemma-3-4B | +0.4 | +0.7 |
| Gemma-3-12B | +1.0 | +0.9 |
In both cases, these datasets are used only to derive the cognitive ordering and localize function-associated heads; their answers are never used to update the model or train a routing policy. We will document this overlap in the revision.
[1] Ma, X., Wang, J., Jiang, Y., Erfani, S., Liu, T., & Bailey, J. Cognitive mirrors: Exploring the diverse functional roles of attention heads in llm reasoning. NeurIPS 2025.
Thank you for your rebuttal
Thank you for your detailed rebuttal, which has addressed most of my concerns. Therefore, I will raise my score to 3.
Response to Reviewer
Dear Reviewer,
We deeply appreciate the time and effort you have dedicated to reviewing our work. Below, we address the questions you raised.
W. The evidence does not isolate cognitive ordering as the source of the gains. The routing criterion relies on correlational head attribution and human first-occurrence order, neither of which shows that a target requires the source's output; and because injections enter the shared residual stream, the gains may reflect generic activation feedback rather than modules composed in a better order. Table~6 does not resolve this: permuting function labels changes the source--target connections and hop count, so a control that matches the routing graph and recomputation budget while changing only the execution order is needed.
Importantly, the injected information is intended to influence latent computation (subsequent attention heads and MLPs), enabling the model to explore more effective reasoning pathways by coordinating existing cognitive functions. The case studies in Appendix A.4 further illustrate this effect: MoCoC promotes a more coherent progression from problem understanding to downstream reasoning, whereas mismatched routing can disrupt this functional sequence.
To further examine this motivation, we grouped the 572 CogQA training questions by their number of ordering mismatches (see the table blow). Baseline accuracy (Llama-3.1-8B) decreases from 82.4% (no mismatch) to 74.9% (one) and 73.0% (two or more). The trend also holds across source datasets: datasets with more mismatches are consistently harder, and the mean mismatch count and accuracy are perfectly inversely ranked across the five sources (Spearman ρ = −1.0). Within individual datasets the pattern is less uniform: four of the five sources show lower accuracy on mismatched questions, while ECQA does not. Because the number of mismatches depends on which cognitive functions a task requires, mismatch frequency and task type are intertwined. We therefore regard this as correlational evidence that motivates MoCoC, not as a causal claim.
Table 1. Baseline accuracy (Llama-3.1-8B, LLM-judged) on the 572 CogQA training questions.
(a) By number of cognitive-ordering mismatches.
(b) By source dataset, with the mean mismatch count per question and accuracy for questions without (=0) and with (≥1) mismatches.
Additional empirical validation: To distinguish cognitive-function dependencies from generic cross-layer feedback, we introduce three controls: (i) topology-matched random routing, which preserves the routing graph and computational budget but replaces function-associated heads with equally many random heads from the same layers; (ii) norm-matched routing, which replaces captured signals with random vectors of equal norm; (iii) layer-pair routing, which preserves the number of hops, captures, and injections but randomizes source–target layer pairs.
As shown in following table, MoCoC with cognitive heads achieves the best average accuracy (56.6%), with clear gains on LogiQA (+3.2) and WinoGrande (+2.5). All three controls remove this benefit. Norm-matched injection stays close to the baseline (55.1%, +0.2), and topology-matched routing is slightly below it (54.8%, −0.1). Random layer pairs degrades performance on all three benchmarks (53.3%, −1.6), indicating that arbitrary cross-layer feedback may be harmful. Together, these results show that the improvement depends on both which connections are routed, as determined by the cognitive ordering, and what is routed, namely the signals of the function-associated heads; it is not explained by injecting a perturbation of the same magnitude.
Table 2. Accuracy (%) of Llama-3.1-8B on three reasoning benchmarks.