acceptodds
Under review as a conference paper at ICLR 2027

LoadCalib: Cognitive-Load-Calibrated Chain-of-Thought Synthesis for Small Language Models

Abstract

Chain-of-thought (CoT) distillation from a large teacher into a small student is typically performed with a uniform verbosity: every problem receives the same style of reasoning trace, regardless of its difficulty or of the student's capacity. We show this one-size-fits-all supervision is measurably suboptimal and propose LoadCalib, which calibrates the cognitive load of teacher CoT to (i) the difficulty of each problem and (ii) the measured capacity of the target student. LoadCalib measures a capacity matrix (the student's accuracy on difficulty bin when distilled with verbosity tier ) on a held-out split, derives a per-difficulty policy with a sufficiency rule (the cheapest tier within of the best), and refines it with a short closed loop rewarded by the student's downstream accuracy. On GSM8K with Qwen2.5 students of 0.5B/1.5B/3B parameters, the oracle-difficulty variant (LoadCalib-oracle) reaches the best uniform tier's accuracy at 3B while using 16% fewer reasoning tokens ( points, not significant; the advantage over the shortest tier is larger and significant, points, paired bootstrap ), and trades 1.1–1.5 accuracy points for 10–12% token savings at 0.5B/1.5B (the oracle variant 1.1–1.7 points for 10–14%). Reverse-calibration and random-tier controls confirm the gains come from calibrated assignment rather than verbosity variance, the variant ordering survives a second training mix and seed changes, and only mid-capacity students show per-difficulty tier differentiation: larger students prefer shorter CoT on easy problems. The deployable learned-difficulty variant (LoadCalib) matches the oracle at 0.5B/1.5B and trails it by 1.2 points at 3B.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
chain-of-thought distillation, small language models, cognitive load, token efficiency, reasoning synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_2gi5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoadCalib, a framework that adjusts the verbosity of teacher-generated chain-of-thought (CoT) reasoning based on problem difficulty and student model capacity. The method first constructs a capacity matrix by evaluating different CoT verbosity levels, then selects an appropriate level for each difficulty category. The authors further introduce a closed-loop calibration process to refine the selection policy using student performance feedback. Experiments with Qwen2.5 models of different sizes show that LoadCalib can reduce generated tokens while maintaining comparable accuracy in some settings.

Critical strengths
  1. The paper investigates an interesting problem in CoT distillation, particularly how to adapt reasoning length to different problem difficulties and student capacities.

  2. The proposed framework considers both problem difficulty and student capacity when selecting CoT verbosity, providing an intuitive approach to improving distillation efficiency.

  3. The paper presents experiments across multiple student model sizes, including ablation studies and comparisons with different CoT verbosity settings.

Critical weaknesses
  1. The contribution of token reduction is not sufficiently convincing. Shorter CoT traces naturally reduce supervision length, and the paper does not clearly establish the advantages of its approach over existing efficient reasoning methods, particularly fast/slow thinking strategies. Since token efficiency is a central motivation of this work, comparisons with relevant approaches would help better demonstrate its contribution.

  2. The experiments are limited to the relatively outdated Qwen2.5 model family. More recent models, such as Qwen3 and Qwen3.5, support explicit thinking modes and can generate substantially longer reasoning traces, making them particularly relevant for studying reasoning efficiency. Evaluations on modern thinking-enabled models would strengthen the practical relevance and generalizability of the proposed approach.

  3. The proposed framework introduces considerable computational overhead, including training multiple student models to construct the capacity matrix and performing additional training during closed-loop calibration. However, the reported token savings are relatively modest. It remains unclear whether these savings justify the additional computational cost. A more comprehensive analysis of the overall training and inference costs would be valuable.

  4. The empirical improvements are limited. In most settings, LoadCalib reduces token usage at the expense of some accuracy. Even for the 3B model, the accuracy improvement over Uniform-medium is not statistically significant. Furthermore, the closed-loop calibration provides only marginal additional gains. These results raise concerns about whether the proposed framework offers a meaningful advantage over simpler alternatives.

  5. The difficulty prediction component is not sufficiently convincing. The learned predictor achieves only 56.5% classification accuracy and relies on heuristic difficulty labels. Its reliability on more complex problems or different domains remains unclear. It would be helpful to investigate how the quality of difficulty estimation affects the overall performance of LoadCalib.

Questions for authors

See weaknesses!

Reviewer_aiddOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how the verbosity of teacher-generated chain-of-thought traces should vary with problem difficulty and student model capacity. LoadCalib first trains students with uniformly short, medium, or long traces and measures their accuracy across four difficulty bins. It then assigns each bin the least costly verbosity tier whose validation accuracy is within two percentage points of the best tier, with an optional feedback loop that retrains and updates the assignment. Experiments with Qwen2.5 students at 0.5B, 1.5B, and 3B parameters evaluate accuracy and generated-token cost, primarily on GSM8K. In the mixed-data setting, the heuristic-difficulty variant achieves 76.3% accuracy at 312 tokens for the 3B student, compared with 75.7% at 373 tokens for uniform medium traces.

Critical strengths
  1. Student-grounded supervision design. The method chooses trace verbosity using the target student’s measured downstream performance rather than relying solely on a teacher’s difficulty judgment. This makes the proposed calibration rule concrete and empirically testable.

  2. Useful controls for the efficiency claim. The paper compares calibrated assignments with uniform tiers, random-tier assignments, and reversed assignments.

Critical weaknesses
  1. Inconsistencies between reported results and textual descriptions. Table 1 reports a medium/short/medium/short policy for the 3B model at δ=0.02\delta=0.02, whereas Section 4.7 claims that the same threshold yields an all-short policy. Similar discrepancies appear for the 1.5B model. Furthermore, the abstract claims that only medium-capacity students exhibit difficulty-dependent length variation, while Section 4.2 describes heterogeneous policies for the 0.5B and 3B models and a nearly uniform policy for the 1.5B model.

  2. Limited evidence for accuracy improvements. For the 3B model, the proposed method improves accuracy over the all-medium baseline by only 0.6 percentage points, with a reported p=0.54, indicating no statistically significant improvement. The largest accuracy gains are obtained against the weaker all-short baseline.

  3. Incomplete end-to-end efficiency evaluation. The paper primarily measures efficiency through inference-time generation tokens, without accounting for the computational costs of training the three probing models, regenerating training data, and performing iterative fine-tuning.

Questions for authors
  1. Could the authors clarify the training protocol, difficulty allocator, and final four-bucket length policy used in each table? In particular, why do Table 1 and Section 4.7 report different policies under the same δ\delta? After resolving these inconsistencies, which main results would require reinterpretation?

  2. What is the total computational cost of the method when accounting for capacity probing, teacher data generation, and multiple rounds of fine-tuning? Approximately how many inference requests would be needed for the generation-token savings to offset these upfront costs?

Reviewer_wYykOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes LoadCalib, a framework for calibrating the verbosity of teacher-generated chain-of-thought (CoT) traces according to problem difficulty and student capacity. It constructs student-specific capacity matrices, selects verbosity tiers through a sufficiency rule, and refines the resulting policy using downstream validation feedback. Experiments on mathematical reasoning benchmarks demonstrate accuracy–efficiency trade-offs across different student model sizes.

Critical strengths

The paper addresses a practically relevant problem of improving the efficiency of CoT distillation for small language models.

Critical weaknesses
  1. Limited methodological novelty. The central idea of adapting CoT verbosity according to problem difficulty has already been extensively explored in prior work, such as LiteCoT [1] and Less is More Tokens [2]. While LoadCalib additionally incorporates student-specific performance feedback, its core methodology primarily consists of evaluating a small set of predefined verbosity tiers, selecting the most cost-efficient tier based on validation accuracy, and iteratively adjusting these assignments (§3). These components largely resemble conventional hyperparameter selection and validation-driven optimization rather than a fundamentally new distillation approach. Moreover, the proposed cognitive-load calibration is operationalized solely through trace verbosity, without introducing a distinct measure of cognitive load. Overall, the methodological contribution appears incremental, and the paper does not sufficiently establish its novelty beyond existing difficulty-aware CoT distillation approaches.

  2. Concerns regarding formatting and overall completeness. In my opinion, the paper would benefit from further refinement and polishing. I would also appreciate it if the Area Chair could verify whether the manuscript complies with the official ICLR 2027 formatting requirements and whether any violations constitute grounds for desk rejection.

[1] Wu et al., Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting, 2025.

[2] Waheed et al., Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation, 2025.

Questions for authors

see weakness section

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.