acceptodds
Under review as a conference paper at ICLR 2027

Steering Large Language Models via Residual Cognitive Demand Profiling

Abstract

Recent meta-reasoning frameworks improve LLM reasoning by wrapping chain-of-thought generation in an iterative control loop, allowing more effective backtracking, termination of reasoning loops, and injection of promising reasoning patterns, among other strategy adjustments. Despite promising results, methods often rely on backward-looking reward functions, utilize coarse search actions, or require additional reasoning controller training requiring many-shot supervision. We introduce Cognitive Demand Steering (CDS), a training-free meta-reasoning framework equipped with residual demand assessment: at each step, an LLM-based progress evaluator characterizes the residual reasoning required to arrive at a solution rather than merely evaluating the previous step. This allows a meta-controller to select reasoning interventions comprising both general-purpose exemplars and actions (e.g., general guidance for quantitative reasoning) that directly tackle this forward-looking demand signal. This shift eliminates the need for any trained component while enabling zero-shot transfer across models and tasks with no adaptation. Rather than relying on coarse characterizations, we employ cognitive scales to both design interventions as well as profile initial problem complexity and residual demand signal over 16 dimensions motivated by cognitive science (e.g., attention and scan, learning and abstraction, spatio-physical reasoning), giving the controller a fine-grained vocabulary for diagnosing. Averaged across three frontier LLMs and six reasoning benchmarks, CDS improves accuracy by 21.9% over direct calls and 9% over standard CoT reasoning, with the largest gains on difficult mathematics and coding tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, reasoning, cognitive demand

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kymMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cognitive Demand Steering (CDS), a training-free adaptive meta-reasoning framework. The main idea is to continuously track the unresolved cognitive demands, uncertainty, and reasoning risks during inference, and use these signals to adapt the next reasoning action and compute allocation. The key contribution is a fine-grained residual cognitive-demand representation that replaces coarser discrete reasoning-mode control, combined with progress evaluation, effort scheduling, and a controller-worker loop. The method is evaluated on mathematics, scientific QA, and code generation benchmarks across several frontier LLMs, where it achieves stronger average accuracy than the compared baselines, with the largest gains on harder tasks such as AIME and LiveCodeBench-Hard.

Critical strengths

1.The paper proposes a clear and intuitively motivated control signal for adaptive reasoning. A key strength is the use of residual cognitive demand as a forward-looking state representation, rather than relying only on coarse discrete actions or retrospective feedback. This gives the controller a more structured way to decide what kind of reasoning is still needed, which is conceptually useful for adaptive inference. 2.The method is training-free and transfers across multiple models and task types. CDS does not require task-specific controller training or parameter updates, and the paper evaluates it across several frontier LLMs and diverse benchmarks covering mathematics, scientific QA, and code generation. This broad applicability is practically important and makes the framework easier to deploy than methods that require additional supervised training.

Critical weaknesses

1.The contribution of cognitive-demand tracking itself is not fully isolated.CDS combines demand profiling, progress evaluation, risk estimation, verification, effort scheduling, and iterative controller-worker reasoning. Although the paper includes a demand ablation, the average gain of full CDS over the ablated variant is relatively small. This makes it difficult to determine how much of the improvement comes specifically from the proposed cognitive-demand representation rather than from additional iterative reasoning and control. 2.CDS can invoke multiple reasoning rounds and several LLM-based components within each round. The paper discusses iteration scaling, but does not provide a sufficiently detailed matched-budget comparison in terms of tokens, latency, or API calls against the strongest baselines. This is important because the practical value of the method depends on whether its accuracy gains justify the additional inference-time cost. 3.The gains are concentrated on harder tasks and are less consistent on easier settings. The paper itself shows that CDS provides only small gains on easier coding tasks and can even slightly underperform direct inference in some cases. This suggests that the framework may not be uniformly beneficial and that its effectiveness depends strongly on task difficulty, which somewhat limits the generality of the claims.

Questions for authors

1.Under matched total token and LLM-call budgets, does CDS still outperform Direct, CoT, ToT, and Meta-Reasoner? Since CDS involves multiple LLM-based components and potentially many reasoning rounds, could the authors report the actual number of model calls, generated tokens, latency, and estimated inference cost for each method? 2.Could the authors provide more fine-grained ablations that separately remove or control for demand profiling, residual-demand assessment, adaptive effort scheduling, verification targets, early stopping, and controller-worker interaction? Reporting multiple seeds or confidence intervals would also help determine whether the observed improvements are robust. This is important because the current demand ablation changes several aspects of the framework simultaneously. 3.How reliable are the residual-demand scores and the resulting stopping decisions? Could an external verifier or human annotation evaluate whether the estimated demand actually corresponds to unresolved reasoning requirements, as well as the frequency of erroneous early stopping or unnecessary continuation? This would help establish whether residual demand is a meaningful control signal rather than merely an internally generated heuristic. 4.Could the authors clarify the exact model/API configuration used for each controller, evaluator, assessor, worker, and baseline, including prompts, temperature, token limits, and stopping rules? In addition, please reconcile the abstract’s claim of a 21.9% improvement over direct calls with the values reported in Table 1, and clarify the apparent GPT-5.5/GPT-5.4 inconsistency between Figure 2 and the main experimental description.

Reviewer_bSDLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces Cognitive Demand Steering (CDS), a training-free framework that guides LLM reasoning by assessing what remains to be solved at each step. It uses cognitive dimensions, such as logical and quantitative reasoning, to identify unresolved demands and guide subsequent reasoning. Experiments on three LLMs across mathematics, science, and coding benchmarks show higher average accuracy than the evaluated baselines.

Critical strengths
  1. The framework is training-free and provides a concrete way to incorporate structured descriptions of unresolved reasoning into inference-time control.
  2. Evaluation across three backbone models and multiple task families is useful. The reported accuracy gains, particularly on AIME, suggest that the overall system merits further investigation.
Critical weaknesses
  1. The benefit of residual-demand tracking is unclear. The ablation in Appendix A.1 removes several components at once, including demand profiling, demand-based guidance, and adaptive effort selection, so it does not isolate the proposed contribution. The average gains over this ablation are small: 0.26, 0.52, and 0.90 percentage points, with no uncertainty estimates reported. Figure 3 also shows that low residual demand can accompany incorrect reasoning. It remains unclear whether these scores provide useful information beyond standard self-critique and uncertainty estimates.

  2. The accuracy gains are not evaluated against their computational cost. CDS makes multiple LLM calls per round and allows up to 12 rounds, but the paper provides no complete cost comparison with the baselines. Its average gains over Meta-Reasoner-D are around 2–2.4 percentage points. Without token usage or comparisons at matched budgets, it is difficult to judge whether these gains justify the additional computation. Figure 2 reports reasoning rounds, which do not capture the overhead of evaluation and control.

Questions for authors
  1. Can you isolate residual-demand tracking while holding guidance, verification, and resource budgets comparable, and report paired performance differences with statistical uncertainty?

  2. Do residual-demand scores predict correctness or improvement from another round beyond ordinary uncertainty and critique? What are the rates of falsely declaring completion and failing to detect incorrect reasoning?

  3. What are the total inference costs, counting all profiling, evaluation, controller, worker, and finalization calls? Does CDS retain its advantage at matched total budgets?

  4. Figure 1 identifies the pipeline components but does not sufficiently explain which states are updated at each stage or how residual demand changes the controller's decisions. Can you clarify the exact update order, reconcile it with Section 3.1 and Algorithm 1, and provide a worked example showing how demand scores translate into an action and a concrete reasoning budget?

Reviewer_uNXiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Cognitive Demand Steering (CDS), a training-free meta-reasoning framework designed to improve LLM reasoning through adaptive cognitive-demand tracking. Unlike existing action-centric approaches, CDS characterizes reasoning requirements using 16 cognitive dimensions and dynamically estimates residual demands throughout the reasoning process. Based on these estimates, the framework adaptively selects reasoning interventions, allocates computational effort, and performs verification to guide the reasoning trajectory. The authors evaluate CDS on three frontier LLMs across six reasoning benchmarks covering mathematics, scientific question answering, and code generation. Experimental results show that CDS achieves the highest average accuracy among the evaluated methods, with particularly notable improvements on challenging mathematical and coding tasks. The paper further investigates iteration scaling and presents a case study to illustrate how CDS can identify reasoning errors and adapt its reasoning strategy.

Critical strengths
  1. The paper proposes a training-free meta-reasoning framework that explicitly tracks residual cognitive demands across multiple dimensions (Sections 3.1–3.2). Compared with conventional action-centric approaches, this provides a more fine-grained and interpretable mechanism for guiding reasoning without additional controller training. This is a meaningful design contribution with potential applicability across different models and tasks.

  2. The experiments cover three LLMs and six reasoning benchmarks, with CDS achieving the highest average accuracy across all three models (Table 1, Section 4.2). The substantial improvements on challenging mathematical and coding tasks are particularly encouraging, suggesting that adaptive reasoning can improve performance on complex problems. The consistency of the average gains across models strengthens the practical relevance of the proposed framework.

Critical weaknesses
  1. Section 4.1 and Table 1 report substantial performance improvements over existing methods, but the paper lacks comparisons under matched computational budgets, including total LLM calls, token consumption, and inference latency. Since CDS involves multiple LLM-based components and iterative reasoning, it remains unclear whether the improvements result from the proposed method or simply additional computation. This significantly weakens the validity of the experimental conclusions.

  2. The central contribution of CDS is residual cognitive-demand tracking (Sections 3.1–3.2), but the experiments do not convincingly isolate its effectiveness. The iteration-scaling results in Figure 2 primarily demonstrate the benefits of additional reasoning rounds, while the case study in Figure 3 provides only anecdotal evidence. Without controlled comparisons against an otherwise identical framework without demand tracking, it is difficult to determine whether the improvements originate from the proposed demand-guided mechanism or generic iterative reasoning. This weakens the empirical justification for the paper's core contribution.

Questions for authors
  1. Section 1 attributes reasoning failures to a mismatch between the reasoning process and task-specific cognitive demands. Is there empirical evidence that this mismatch is a major source of reasoning failures, rather than insufficient computation or inadequate verification?
  2. Sections 2 and 3.2 extend cognitive scales originally designed for task-level characterization to dynamically estimate residual reasoning demands. What justifies this extension, and how can the authors establish that these estimated residual demands reliably reflect the actual unresolved reasoning requirements?
  3. Section 3.2 (p. 5) selects the top four cognitive dimensions, while Algorithm 1 (Appendix A.5, p. 20) subsequently focuses on three. How sensitive is CDS to the number and selection of active dimensions? Could initially excluded dimensions become important during reasoning, and how would CDS handle them?
  4. Figure 1 (p. 3), Section 3.1 (p. 4), and Algorithm 1 (p. 20) appear to describe different execution orders for residual-demand assessment. Could the authors clarify the exact information flow? Furthermore, how are the generated verification targets reliably validated when external tool use is prohibited (Section 4.1, p. 7)?
  5. This paper compares CDS with Direct, CoT, ToT, and Meta-Reasoner. Are these methods evaluated under comparable inference budgets, including total LLM calls, reasoning tokens, and computational costs? Without such controls, how can the improvements be attributed to CDS rather than additional computation?
  6. Could the authors provide accuracy-versus-computation curves under matched token or latency budgets, and compare CDS against a prompt-matched controller-worker framework using random or fixed demand profiles? Such experiments would help isolate the contribution of informative residual-demand signals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.