acceptodds
Withdrawn from ICLR 2027

Overtaught, Underlearned: Learner-Centered Distillation from Multi-Agent Discussion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-agent discussion produces reasoning collaboratively, while deployment still relies on a smaller student that must answer alone. Full-graph distillation transfers such discussions by treating each correct solution as a separate training target, but what teachers provide need not match what the student learns. This mismatch appears in two ways: several correct solutions may amount to the same lesson in the student’s representation (over-teaching), while some training questions remain missed after learning (under-learning). We quantify both with rates that are high on discussion graphs. To address them, we introduce FAR (Fold–Attempt–Revisit), a learner-centered distillation procedure that organizes supervision around the student’s own state. Before training, FAR folds solutions within each discussion state into representative lessons. After training, the student’s own attempts identify missed questions, while poorly represented clusters indicate where folding may have lost useful variation; FAR then writes targeted lessons from retained discussion contexts. Across two students and four reasoning tasks, FAR outperforms full-graph distillation with far fewer training targets and lower under-learning. Further analyses examine the behavior of FAR and the sources of its gains. Overall, the results show that organizing discussion supervision around the student can make distillation both more effective and more efficient.

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge distillation, multi-agent discussion, learner-centered learning, large language models, reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ozD8OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors argue that over-teaching can cause harm to the student model, and present FAR, a pipeline that targets the weakness of a student model, which first identifies what is missing and curates a targeted lesson to teach the student model to learn those missing capabilities.

Critical strengths

Overall, the idea of targeting what capability is still missing in the current student checkpoint is sound and sensible.

Critical weaknesses
  1. There are a lot of design choices in the pipeline, and few of them are validated by experiments. For example, in line 203 "We summarize each solution by frequency-weighted pooling (Arora et al., 2017), using IDF so terms repeated across correct solutions contribute less", it's not clear whether this is the best way to do so. Same for line 213, "the lesson count is its rounded effective rank (Roy & Vetterli, 2007)..." and line 221 "the over-teaching rate".

  2. The main weakness of this paper is that the student models are too weak. The authors use Mistral-7B-Instruct-v0.2 and Qwen1.5-4B-Chat, both are outdated. It is understandable to train small models, but it is necessary to show the effectiveness of the proposed pipeline on models later than Qwen3 / Gemma 3.

  3. Relatedly, many of the benchmarks evaluated are saturated nowadays with Qwen3-4B-Instruct-2507 or newer models.

Questions for authors
  1. The center claim "Repeated correct solutions can over-teach the student while leaving questions unsolved" is not intuitive to me. Like SFT or rejection sampling fine-tuning, using multiple correct solutions to teach the student model shouldn't hurt what the student already knows about that particular problem, and can even improve its overall performance. Is there any reference or empirical result can support this?

  2. It is not very clear what "Initial", "Sustained", "Corrected" mean in Figure 2. How "State-aware geometry" separate these groups? The method section is overly complex, with many equations that might not be necessary, making it harder to follow and less connected to the figure.

  3. The method seems to form a loop with Fold-Attempt-Revisit, but in the figure and text, it seems that the authors only do this loop once. What happens if we iteratively train the student model?

Reviewer_i8yUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FAR, a three-stage framework for distilling multi-agent discussion data into a single student model. Fold selects representative correct solutions, Attempt identifies questions the trained student still misses, and Revisit generates targeted lessons for continued training. Experiments use released ReConcile discussion graphs with Mistral-7B and Qwen1.5-4B students.

Critical strengths
  1. The motivation is clear. Decoupling over-teaching ( where correct solutions repeat a lesson in the student’s representation ) from under-learning ( where the trained student still misses teachable questions ) is a valuable perspective on multi-agent discussion distillation.
Critical weaknesses
  1. The novelty and contemporary relevance are limited. The multi-agent system primarily serves as a data-generation mechanism, while the paper largely repurposes released ReConcile[1] graphs. Moreover, Attempt–Revisit closely resembles FAIR[2]’s “student exam–failure identification–targeted remediation” pipeline, which is not discussed. The evaluation relies on outdated benchmarks such as StrategyQA and outdated students such as Qwen1.5-4B, making it unclear whether the findings remain relevant to current open models and reasoning tasks.

  2. Necessary baselines and ablations are missing. Revisit lacks an equal-budget control that randomly selects questions or sources while using the same frozen-student writer and training-token budget, so its gains cannot be attributed specifically to learner-centered selection. Fold also lacks simple matched controls such as exact deduplication followed by random selection, fixed representatives per state, or clustering without discussion-state partitioning.

[1] ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs, ACL2024 Main

[2] Learning from Committee: Reasoning Distillation from a Mixture of Teachers with Peer-Review, ACL2025 Findings

Questions for authors
  1. How does FAR differ substantively from FAIR, particularly its student-exam, failure-identification, and targeted-remediation pipeline?
  2. Can the authors validate FAR using contemporary models such as Qwen3/Qwen3.5 and more challenging benchmarks covering advanced mathematics, coding, or long-horizon agent tasks to demonstrate its relevance and generalizability in current settings?
  3. Can the authors add equal-budget random-selection controls for Revisit and simple matched selection/partitioning ablations for Fold, to isolate whether the reported gains specifically arise from the proposed learner-centered mechanisms?
Reviewer_bx83OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows how multi-agent discussions can be distilled into a smaller model. Their approach FAR first selects representative solutions, then checks which training questions the student still answers incorrectly. The student then writes additional information using selected discussion and continues training. Experiments cover multiple models and tasks.

Critical strengths

a) The paper addresses a useful problem. Repeated correct solutions are not necessarily useful and also not practically worthwhile.

b) The workflow is understandable and well motivated. Figures 1 and 2 help understand motivation and method clearly.

c) The empirical results are encouraging.

Critical weaknesses

a) The interpretation of over-teaching needs better justification. The paper treats similar solutions as redundant without establishing whether they provide the same learning benefit. Similar explanations may still differ in a crucial step that the student has not understood. Moreover, the reported over-teaching rate counts material removed by the proposed selection procedure, rather than independently showing that it was unnecessary. This weakens both the claim that the paper measures over-teaching and the justification for deciding which solutions do not need separate study.

b) Student-aware rationale selection has already been studied, for example, in MoRSD (Yan et al., EMNLP Findings'25). FAR’s benefits need clearer evidence. Authors must compare against a random or simpler selection baseline to show benefits. Similarly, the Revisit ablations do not compare against rewriting an equally sized set of randomly selected questions.

c) The overall efficiency benefit remains unclear. Table 12 reports training costs for Fold, but not the complete FAR. The paper acknowledges it, but lower Fold costs do not establish that the complete method is computationally cheaper.

d) Important experimental details are missing. The results are described as run means, but the number of runs and variability are not reported. This matters particularly for the smaller transfer improvements. Appendix C.1 (Table 4) also refers to structural and arithmetic acceptance checks without specifying them sufficiently, although these checks determine which generated lessons are used for training.

Questions for authors

Please check weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.