acceptodds
Under review as a conference paper at ICLR 2027

From Intervention to Skill: Evolving Small–Large Language Model Collaboration

Abstract

Small–large language model collaboration improves inference efficiency by assigning routine decisions to a smaller model and invoking a stronger model when additional capability is needed. Existing approaches primarily optimize this allocation for the current interaction, leaving the value of a large-model intervention largely transient. We study whether past interventions can instead enable collaboration to evolve. We introduce Intervention-to-Skill (I2S), which converts execution-supported large-model interventions into reusable skills. I2S localizes the decision opportunity at which stronger assistance is requested, grounds its effect in environment feedback, and uses trajectory context to identify a reusable procedural residual. The resulting skills persist across tasks and evolve with new evidence, allowing the effective small-model policy to improve while both acting models remain fixed. Across ALFWorld, WebShop, and ScienceWorld with Qwen3-8B paired with Qwen3-32B, DeepSeek-V4-Flash, and Gemini-2.5-Flash, I2S consistently improves task quality over strong routing and skill-based baselines. Compared with SkillGen, task-quality gains reach 27.1 score points and weighted token usage is reduced by up to 63.2%. As skills accumulate, task quality increases while large-model involvement decreases, and acquired skills are repeatedly reused in later small-model decisions. These results establish small–large collaboration as a cumulative adaptation process in which previous assistance reshapes the future division of labor between models. Code is available in the Supplementary Material.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Small–Large Model Collaboration, Skill Learning, Language Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6rDaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Intervention-to-Skill (I2S), a framework that evolves small-large language model collaboration in interactive tasks. Rather than merely routing requests dynamically, I2S lets the small language model (SLM) attempt tasks first; when confidence is low, a large language model (LLM) intervenes. Crucially, I2S extracts the successful LLM interventions and grounds them with environment feedback to formulate reusable, structured procedural "skills" (JSON cards). These skills are then added to a shared external state, enabling the SLM to handle similar difficulties independently in future tasks and cumulatively reducing reliance on the expensive LLM over a deployment stream.

Critical strengths
  • Novel Shift from Transient Routing to Cumulative Adaptation: While most existing literature focuses on stateless query-level or step-level routing, this work pioneers the concept of converting transient LLM interventions into persistent, cross-task skills. This is a highly practical and insightful approach for continuous agent deployment.
  • Rigorous Anti-Hallucination Skill Induction: The framework does not rely on free-form text summaries for skills. Instead, it utilizes strict JSON schema templating (ω=⟨Iω,Pω,Oω⟩\omega = \langle I_\omega, P_\omega, O_\omega \rangle) coupled with a physical Evidence-grounded admission mechanism (CheckEvidence). This effectively prevents the LLM from hallucinating generic, ungrounded task recipes.
Critical weaknesses
  • Presentation Ambiguity Regarding Confidence Scores: In Figure 2, the pipeline illustrates a continuous confidence score (e.g., c=0.18c=0.18), yet the methodology implies the LLM outputs discrete labels (Prompt D.2 outputs CONFIDENT or UNCERTAIN). While the mechanism (applying softmax over target token logits) is explained deep in Appendix A.1, its omission from the main text (Section 4.2) causes significant confusion regarding how the continuous threshold (γ=0.99\gamma=0.99) is actually computed during inference.
  • Cost Inversion in Complex Environments: In Table 1 and Figure C.1, the complete-stream weighted token usage (WW) for I2S with DeepSeek-V4-Flash on ScienceWorld (31.631.6k) is astonishingly higher than simply using the Large-Only model (29.029.0k). The "propose-first" design incurs a massive cost penalty when the SLM is highly incompetent for a complex task, as the system must pay for the SLM's generation, the confidence scoring, and the subsequent LLM intervention at nearly every step. The paper lacks a critical analysis of this cost inversion phenomenon.
  • Unexplored Lower Bound of SLM Capabilities: The framework relies on the SLM (Qwen3-8B) to accurately parse long contexts containing complex JSON skill cards (skill_refs), interpret analysis rationales, and adhere to strict action grammars. An 8B model is already highly capable. It remains entirely unclear whether this I2S pipeline would completely collapse if paired with a genuinely "small" model (e.g., 1.5B or 3B parameters) due to catastrophic forgetting or instruction-following failures.
Questions for authors
  1. Clarification on Confidence Calculation in Main Text: Could you briefly add a sentence in Section 4.2 explaining that the continuous confidence score cc is derived from the softmax of the log-probabilities of the CONFIDENT and UNCERTAIN tokens? This would immediately resolve the apparent contradiction between Figure 2 and Prompt D.2.
  2. Analysis of Cost Inversion in ScienceWorld: For the DeepSeek-V4-Flash + ScienceWorld setup, I2S consumes more weighted tokens (31.631.6k) than the Large-Only baseline (29.029.0k). Does this suggest that when the capability gap between the SLM and the task complexity is too large, the "SLM proposes first + confidence scoring" mechanism becomes an active liability? Would a "fail-fast" or dynamic thresholding mechanism be necessary to prevent this triple-billing of tokens in highly complex environments?
  3. Capability Requirements for the Small Model: The framework expects the SLM to parse and utilize injected JSON procedural skills effectively. Qwen3-8B is relatively powerful. Have you experimented with truly small models (e.g., 1B-3B parameters like Qwen2.5-1.5B or LLaMA-3.2-3B)? Is there a fundamental capability threshold below which the SLM fails to utilize the induced skills, thereby causing the I2S loop to break down? Adding a smaller model ablation would significantly strengthen the paper's claims about "Small-Large" collaboration.
Reviewer_bd1KOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper explores small-large model collaboration in AI agents, where a small model (Qwen3-8B) acts by default and a larger model intervenes when needed. A trained confidence scorer decides when the large model intervenes, and I2S converts those interventions into reusable skill cards. After task completion, interventions confirmed by the environment yield skill cards that are continually updated across tasks. Evaluated on ALFWorld, WebShop, and ScienceWorld with Qwen3-32B, DeepSeek-V4-Flash, and Gemini-2.5-Flash, I2S achieves top or tied-top performance across all nine settings compared to routing (AutoMix) and skill-memory baselines (ExpeL, Dynamic Cheatsheet, SkillGen). The paper also reports lower weighted token use than most baselines, decreasing large-model involvement, and repeated skill reuse.

Critical strengths

Consistent performance: I2S ties or achieves the best score across all nine settings under identical task orderings. Furthermore, token accounting includes overhead from skill induction and confidence scoring.

Critical weaknesses
  1. Unproven contribution of skills: The paper fails to demonstrate that skills drive performance gains.
  • The sole ablation (Tab C.2) lacks a calibration-only control. Comparing skills on vs. off (w/o calibration) shows that adding skills slightly reduces performance while increasing token usage by about 40%. Thus, A. B.3 incorrectly claims these controls isolate skill contributions.
  • This ablation is conducted in only one setting where several baselines already approach 100%.
  • Results rely on single-run sampled decoding without variance reporting. Small-Only (same model, same tasks) varies by up to about 5 pts across backends (ScienceWorld), exceeding several of I2S's winning margins.
  • Gains may stem from the confidence gate rather than skills. Furthermore, A. B.3 omits deferral mechanisms for ExpeL, Cheatsheet, and SkillGen, which may confound controller and skill effects.
  1. Weak efficiency claims and missing prior work.
  • I2S uses fewer weighted tokens than Large-Only in only 4 of 9 settings (e.g., Large-Only with Gemini is cheaper across all benchmarks). The efficiency metric counts output tokens only and applies the 8B/32B weight from the Qwen pair unchanged to API models.
  • Converting strong-model assistance into reusable weak-model guidance is established prior work (Inter-Cascade, Wu et al. 2025, on reasoning benchmarks; Inference-Time Distillation, Sarukkai et al. 2025, on ALFWorld), yet neither is cited or compared.
Questions for authors
  1. Can you add a calibration-only expt (confidence gate, empty skill state), at least on ScienceWorld where gains are largest? How do you reconcile Tab. C.2, where skills hurt without calibration, with the claim that skills drive the gains?
  2. How do ExpeL, Cheatsheet and SkillGen decide when to call the large model? Do they get your confidence gate and recovery triggers?
  3. What is the run-to-run variance per setting, and which gaps survive a paired test?
  4. With input tokens and dollar cost included, is I2S cheaper than Large-Only? Are Gemini and DeepSeek reasoning tokens counted?
  5. How does I2S differ from Inter-Cascade, and how does it compare with Inference-Time Distillation on ALFWorld?
Reviewer_3Zy6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at a large-small model collaboration where the answers provided by the large model are turned into re-usable skills that allow the small model to avoid calling the large model. The entire system has multiple parts for deciding whether to route based on the small model's confidence, which is conditioned on its available skills and another part that turns the large model's answer it something reusable. The experiments use three benchmarks and three combinations of small-large models, showing that the system is more accurate than using the large model (and other benchmarks) while using fewer "adjusted tokens". Multiple ablations and analyses are included in the appendix.

Critical strengths
  • Clever idea to reposition the large-small model design to an iterative setup where large-model invocations can be repacked as re-usable skills

  • Diverse evaluation setup with multiple pairs from different model families and multiple benchmarks

  • Good ablation and component analyses (e.g., the confidence analysis)

Critical weaknesses
  • The current evaluation oversimplifies the costs for I2S by focusing on weighted tokens and leaving out the costs of the intermediate parts of I2S. For these multi-model systems, often practitioners care about (1) speed, where the small model is faster or (2) cost, where the small model is cheaper. I2S requires both the confidence scoring and the skill induction. Both of these are LLM based and I could not find any details of which models were used (though the prompts are helpfully provided in the appendix). These systems should count towards the total cost of I2S. If you need to do some weighting (which seems fair), I'd prefer to see something like total cost were you use different cost-per-token estimates for each model, derived from a real source like OpenRouter or your preferred multi-LLM serving end point. You could also use latency too, which would be helpful to see.

  • The paper has a lot of details, but somehow leaves out key implementation details on the other pieces in favor of formula-based descriptions of what is done. I appreciate the goal of technical precision, but, to me, this is getting in the way of seeing how the actual system is working.

  • Evaluations use three benchmarks, but from what I understand of them and can tell from B.1, these are fairly closed-world tasks where there is a limited range of possible questions types that might be asked of I2S (WebShop might be the most general?). This should give I2S an advantage since there's higher opportunity for skill generation and re-use. However, I'm not sure how well it would work in more realistic workflows where there's a higher variance in the input question/task type. Some kind of analysis on this might help give a sense of where/when I2S will generalize

Questions for authors
  • I was pretty surprised that the teacher model somehow has a lower score in all the evaluations than I2S (Figure 3, all cells). Given that the smaller model is likely worse overall, and the teacher is being used to raise its capabilities, do the authors have any intuition or analysis for how the combination somehow results in stronger performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.