acceptodds
Under review as a conference paper at ICLR 2027

From Teacher Imitation to Failure Diagnosis: Causality-Inspired Test-Time Evolution of Small Language Models with LLM Feedback

Abstract

Test-time evolution of small language models (SLMs) with large language model (LLM) guidance has emerged as an important paradigm for LLM-SLM collaboration. Existing methods typically distill complete LLM-generated solutions into SLMs, while recent approaches selectively transfer knowledge at difficult tokens or intermediate reasoning steps identified from the SLM's reasoning state. Despite differing in what to transfer, these methods largely follow the same paradigm: when an SLM fails, it imitates the correct solution or reasoning provided by a stronger LLM. However, substantial differences in model capacity, training data, and acquired knowledge may make such direct imitation difficult for SLMs to effectively absorb. We take a different perspective: rather than only showing an SLM what the correct solution is, the LLM should first diagnose why the SLM fails. Such diagnosis serves two purposes. First, it guides the LLM to minimally revise the SLM's erroneous reasoning, preserving what the SLM already does correctly while correcting only failure-relevant parts. This provides a learning target closer to the SLM's current reasoning capability than an independently generated LLM solution, facilitating more effective knowledge transfer. Second, the diagnosis itself provides an additional learning signal, enabling the SLM to learn not only how its reasoning should be corrected but also why it failed. Based on this insight, we model LLM-guided SLM evolution from a causal perspective and propose CausalBridge. Reasoning State Probing samples and clusters multiple reasoning trajectories to characterize the SLM's reasoning state. Causal Diagnostic Mediation diagnoses the underlying failures and generates minimal, diagnosis-guided revisions. Finally, Mediator-Guided Evolution uses the diagnostic information as a mediator to improve failure-relevant reasoning. Extensive experiments across diverse reasoning tasks show that CausalBridge consistently outperforms both independent SLM evolution and existing LLM-SLM collaborative evolution methods.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Collaborative Evolution, Test-Time Learning, Causal Inference, Deconfounding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MipSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose CausalBridge, a test-time learning framework that uses LLM-generated failure diagnoses to guide minimal corrections of SLM reasoning trajectories, framed through a front-door causal criterion. The method works by sampling and clustering SLM trajectories to identify failure states, querying an LLM to diagnose the primary error and produce a locally revised trajectory, then training the SLM via DPO on diagnosis-conditioned and query-only preference pairs. Experiments on Qwen2.5-7B-Instruct across MedQA, MATH, SATBench, BIG-Bench LD, LogiQA2.0, and FOLIO report 2–7 point improvements over a base SLM, with the method claiming superiority over DSS, LLM2LLM, and SuperCorrect — though the latter three score below the unadapted base on MATH by 3.6–13.3 points. The ablation on 4 benchmarks shows diagnosis-guided LoRA outperforms answer-only and LLM-CoT SFT variants.

Critical strengths
  1. Diagnostic ablation: Table 2 isolates the contribution of diagnostic feedback vs. answer-only vs. full CoT supervision, showing diagnosis-guided LoRA outperforms both on all 4 reported benchmarks (e.g., FOLIO 69.50 vs. 64.00/65.60).

  2. Principled data filtering: The adaptation pipeline (Appendix A.1–A.2) applies structured validation — single primary error, local-edit constraints, copy ratio ≥ 0.55, answer agreement — which is careful engineering that reduces noise in the training signal.

Critical weaknesses
  1. Broken baselines invalidate the main comparison: In Table 1, LLM2LLM scores 49.60 on MATH vs. base 57.50 (−7.9pp), SuperCorrect scores 44.20 (−13.3pp), and DSS scores 53.90 (−3.6pp) — all three degrade the base model. These are offline distillation methods (designed for pre-deployment training with large datasets) dropped into a test-time few-shot setting they were never designed for. Reporting them as baselines makes CausalBridge's 63.80 appear as a 14–20 point improvement when the real gain over the unbroken base is 6.3 points. The comparison is both misconfigured and category-mismatched; it does not test the paper's core claim.

  2. Causal framing is decorative: The front-door criterion (Eq. 3) is stated in Section 3.2 but never computed or enforced — Section 4.4, line 344 explicitly admits "the empirical objective does not estimate the front-door adjustment directly." The actual method is standard DPO on LLM-generated preference pairs. The "mediator" is a prompt engineering choice (ask the LLM to diagnose before correcting) relabeled with causal notation. Removing all causal language would not change any line of the algorithm.

  3. Critically insufficient experimental coverage: Main results (Table 1) use a single SLM (Qwen2.5-7B) and single teacher (DeepSeek-V4-Flash). The generalization experiment (Table 3) covers only SATBench — one of six benchmarks — with marginal gains (Llama-8B: +1.84, Mistral-7B: +1.40, GLM teacher: +1.40). No multi-benchmark generalization across backbones or teachers is shown. No cost-controlled comparison exists: CausalBridge uses LLM API calls during adaptation while self-adaptation baselines (TLM, SRPO, ActiveFT) do not — yet they are compared on accuracy alone. Missing natural baselines: simple self-refinement, best-of-N, or DPO-without-diagnosis ablation at the same LLM token budget.

Questions for authors

The same as above

Reviewer_FdyTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the effectiveness of distillation for small language models. Existing methods let the SLM directly imitate the correct solution from the LLM, while the proposed method CausalBridge first let LLM diagnoses why the SLM fails and then minimally revises the SLM's reasoning. Experiments across diverse reasoning tasks show that CausalBridge outperforms both independent SLM evolution and existing LLM-SLM collaborative evolution methods.

Critical strengths
  1. The motivation is insightful. The reason why an SLM can fail may contain more important information than the wrong answer itself.
  2. The presentation is clear. The paper is easy to follow and well-organized.
Critical weaknesses
  1. The improvements are limited. The results of base SLM in Table 1 are very close to the improved resutls with only 2-3% improvements for MedQA, MATH, and LogiQA2.0. The results of the repeating training runs are not reported.

  2. The cost is not measured. The proposed method involves several additional steps for requesting LLMs. The trade-off between cost and performance should be discussed.

  3. The causal formulation is not used and verified in the method. The front-door adjustment take a large part of the method section but is not estimated finally.

  4. The key claim that the SLM learns why it fails is not supported, since the diagnosis is only given in the prompt and is never generated by the SLM.

Questions for authors
  1. What's the additional cost of the proposed method?
  2. How does this causal design help the overall framework and results?
  3. Why is the result on SATBench 56.50 in Table 1 but 53.40 in Table 3?
Reviewer_P5FuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper is about helping a Small Language Model (SLM) learn from a big one by explaining its mistakes, rather than just handing it the right answer. The authors call the method CausalBridge. SLMs are cheap and run on a phone or laptop, but they make more reasoning mistakes on complex tasks. One common fix is to have a big model teach the small one by having it copy the big model’s full solution. The authors argue that copying often does not work well. Instead of “here is the right solution,” the big model says “here is exactly where and why you went wrong,” then fixes only that part of the small model’s own attempt. This gives two benefits: the corrected version stays close to what the small model wrote, and the explanation is itself a lesson. The method samples several attempts from the small model, has the big model (DeepSeek-V4-Flash) diagnose and minimally correct the wrong ones, and trains the small model (Qwen2.5-7B) on these corrections with preference optimization (DPO). The paper reports gains over test-time learning and teacher-imitation baselines on five of six reasoning benchmarks, and the adapted model needs no cloud calls at deployment.

Critical strengths
  1. A motivated, practical idea: diagnose and minimally fix the student’s own attempt instead of copying the teacher’s solution. This addresses the capacity gap in distillation, and the adapted model needs no cloud calls at deployment.

  2. Consistent gains across tasks, backbones, and teachers. It is best on 5 of 6 benchmarks, and helps Llama and Mistral, as well as Qwen, with three different cloud teachers.”

Critical weaknesses
  1. The paper never isolates diagnosis as the source of the gains. The paper’s contribution is the claim that diagnosis, as a mediator, is what helps. The causal machinery isn’t used, so that claim rests entirely on the experiments, and they don’t separate diagnosis from the training-objective change.

  2. Reported numbers are inconsistent, and the margins are within noise. In the same setting, SATBench with Qwen2.5-7B and DeepSeek-V4-Flash is reported twice with different numbers: Table 1: Base 49.08, Ours 56.50. Table 3: Base 49.60, Ours 53.40. Several margins over the best baseline also fall within the reported ± (e.g., BIG-Bench LD 56.80 ± 3.25 vs 54.80 ± 3.25), and the ± reflects sampled answers from one trained model, not training runs.

  3. “Test-time” is misleading. Corrections are filtered against the support set’s gold labels, so this is supervised few-shot fine-tuning.

Questions for authors
  1. Separate the two training branches. Report results for direct pairs only, diagnosis-conditioned pairs only, and both. Also add a DPO baseline that prefers the full teacher CoT over the small model’s attempt, with no diagnosis. Your method uses DPO while the imitation baselines use SFT, so these comparisons are needed to show that the gains come from diagnosis rather than from preference training.

  2. Baseline tuning and hyperparameters. Report the LoRA rank, learning rate, DPO β, number of sampled trajectories, and the clustering method. Explain how the baselines were tuned; SuperCorrect and LLM2LLM drop MATH well below the base model (44.2 and 49.6 vs 57.5). Give the number of cloud calls and the training pairs per dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.