- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper introduces BeaconAgent, a technique enabling collaboration between Small Language Models (SLMs) and Large Language Models (LLMs). BeaconAgent allows SLM agents to collaborate with more capable LLMs while retaining their role as primary executors. By combining "failure-driven supervised fine-tuning (SFT)" with "capability-aware reinforcement learning (RL)," the system learns where, how, and when to seek assistance through mechanisms such as critical error localization, concise and targeted guidance, and capability-matched help-seeking. Across eleven benchmarks—including unseen code generation—BeaconAgent optimizes the trade-off between performance and cost by minimizing unnecessary LLM interventions while maintaining excellent task performance.
Critical strengths
This paper focuses on the critical and pressing topic of collaboration between SLMs and LLMs. The proposed BeaconAgent method was evaluated across 11 benchmarks, including unseen code generation. The results demonstrate that BeaconAgent can reduce unnecessary calls to LLMs while maintaining excellent task performance.
Critical weaknesses
Existing literature already contains studies addressing the same or similar problems—such as "When2Ask," published at the 2024 Reinforcement Learning Conference (RLC). However, the authors of the paper in question fail to provide a literature review; they neither mention these related studies nor compare BeaconAgent with those methods.
The content of Section 2 (especially Section 2.2) is difficult to understand. It is recommended to refine the text and clarify the meanings of the mathematical symbols.
Questions for authors
See the above "Critical Weaknesses" Section.
4 comments
We sincerely thank the reviewer for the valuable comments and constructive suggestions. We have carefully revised our paper accordingly and resubmitted the updated version. The major revisions are summarized as follows:
(1) Improving the readability of Section 2.2. We have revised Section 2.2 to improve its clarity and readability, providing clearer explanations of the mathematical notations, training procedures, and the overall logic of our proposed method.
(2) Adding discussion and experiments on When2Ask. We have expanded the Related Work section to cite, discuss, and compare When2Ask with BeaconAgent, clarifying the differences in their approaches to learning help-seeking behaviors. Furthermore, we have conducted additional experiments by adapting When2Ask to our experimental setting and comparing it with BeaconAgent across all eleven benchmarks. The results show that When2Ask is a strong baseline, achieving a macro-average accuracy of 53.34% and outperforming all three baselines reported in the main paper, namely AdaSwitch (47.80%), R2vAgent (53.32%), and STEER (48.59%). Nevertheless, BeaconAgent further improves the average accuracy to 57.55%, outperforming When2Ask on ten of eleven benchmarks. Moreover, BeaconAgent provides brief and targeted assistance, with only 49.39 LLM output tokens per assisted trajectory. These results demonstrate the effectiveness of BeaconAgent in achieving stronger task performance through fine-grained help seeking and minimal LLM assistance.
We sincerely appreciate the reviewer's feedback, which has helped us improve the quality and clarity of our manuscript. We look forward to the reviewer's further comments and response.
Thank you for the author's response; it addresses my primary concerns. I have updated my rating to "weak acceptance".
We sincerely thank the reviewer for reconsidering our work and for increasing the score. We greatly appreciate your constructive feedback and thoughtful evaluation, which have helped us further improve the quality of our paper.
Response to Reviewer WTK6
Thank you for your careful review and valuable suggestions. We particularly appreciate your recommendation of When2Ask (Hu et al., 2024), which is an important and valuable study on learning when an agent should interact with an LLM.
(1) Relation to When2Ask. We believe that When2Ask is an important and valuable contribution to learning efficient agent–LLM collaboration, and it provides meaningful insights for our research. We sincerely appreciate the reviewer for bringing this work to our attention. In the revised manuscript, we will cite and discuss When2Ask in the Related Work section and provide a comparison between When2Ask and BeaconAgent.
When2Ask focuses on embodied intelligence scenarios and introduces a Planner–Actor–Mediator architecture, in which a PPO-trained asking policy determines when to request high-level plans from an LLM based on environmental observations. Its experiments in MiniGrid and Habitat demonstrate that learning when to interact with an LLM can effectively reduce unnecessary interactions while maintaining task performance.
BeaconAgent addresses SLM–LLM collaboration in the context of language model agents performing complex reasoning and tool-use tasks. Its key innovation lies in jointly learning where to seek help, how to request targeted assistance, and when to seek help according to the SLM's evolving capabilities. Specifically, failure-driven SFT enables the SLM to identify critical errors and formulate targeted help requests, while capability-aware GRPO aligns help-seeking decisions with its capability boundary. Moreover, BeaconAgent retains the SLM as the primary executor throughout task execution, with the LLM providing only brief, targeted guidance, thereby improving task performance while minimizing reliance on the LLM.
We believe that When2Ask and BeaconAgent represent complementary research efforts that explore efficient agent–LLM collaboration in different application scenarios and from different methodological perspectives. We highly appreciate the contributions of When2Ask and sincerely thank the reviewer for this valuable suggestion, which helps us better position our work within the existing literature.
(2) Clarity of Section 2, particularly Section 2.2. Thank you for pointing out the difficulty in understanding the methodology. We agree that the presentation of Section 2.2 can be improved. In the revised manuscript, we will reorganize this section, explicitly define the mathematical symbols when they first appear, and provide clearer explanations of the corresponding equations. In particular, we will clarify the capability-label construction and dynamic relabeling procedures, the meanings of the rollout-group statistics and reference help-request positions, and the respective roles of the task-outcome reward, repeated-request penalty, and critical-step alignment reward. We will also explain the reward calculation and clipping procedure step by step to make the overall optimization process easier to follow and reproduce.
We sincerely thank the reviewer again for highlighting these important related works and presentation issues. These suggestions will help us better position our contribution and improve the clarity of the paper.