- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies collaboration between a small on-device model and a large cloud model on long-horizon agentic tasks. The paper proposes Sibyl, which treats call_cloud as an action of the small model itself. At each agentic step, the action chosen by the small model can be either a task action or call_cloud, and if it chooses call_cloud, the large cloud model selects the action for the current step on its behalf. Sibyl learns when to call_cloud through a three-stage training procedure, (i) it first trains the small model into a fairly strong standalone agent with RL and OPD, without allowing it to call_cloud, (ii) it then identifies the states where replacing a single action with the cloud model's action turns a failure into a success, and uses them for SFT so that the small model learns call_cloud in these states, and (iii) it finally continues RL with call_cloud, using comparisons among matched states to judge whether a call is worthwhile, while distilling the cloud model's guidance into the small model to reduce future calls. Using small Qwen3 models as the local models and Qwen3.7-Max as the cloud model, the paper compares with agent training methods and routing methods on ALFWorld and WebShop, and claims higher success rates with very few cloud calls.
Critical strengths
None, I cannot identify any strength of this paper. The key idea of the paper is highly similar to uncited prior work, so the novelty is incremental. The experiments can barely support the claimed contributions. They are not only insufficient, but, judging from the results, the experimental setup and implementation also seem unreasonable. When the cloud call ratio is higher, the success rate is lower, which seems to suggest that the cloud model performs even worse than the fine-tuned small model. Therefore, the paper does not seem to have any strength in terms of novelty, contribution, or experiments.
Critical weaknesses
- The key idea of this paper is highly similar to [1], yet the paper neither cites nor discusses it. [1] has already proposed letting a small model treat asking the cloud for help as an action in its own policy, training it with RL to decide when to ask for help, while limiting the number of cloud calls and without requiring an external router. This is exactly the first contribution claimed in this paper. The only difference is that this paper replaces single-turn reasoning with multi-step agentic tasks, but in essence the idea, the contribution, and the method are almost the same. The idea of letting the model itself learn when to defer to a stronger model, without an external router, has also been explored in [2][3], neither of which is cited. In addition, among the three training stages designed in this paper, Stage 1 largely comes from SEED, and the matched-state grouping in Stage 3 is also similar to GiGPO. Therefore, the novelty is incremental.
[1] Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training. ICML 2026.
[2] Learning to Decode Collaboratively with Multiple Language Models. ACL 2024.
[3] Learning to Route LLMs with Confidence Tokens. ICML 2025.
-
I have serious concerns about the correctness of the experimental implementation, and the experimental results cannot show that routing is really useful. (i) Asking for help brings almost no benefit. The cloud call ratio of Sibyl (Qwen3-1.7B) is only about 3% on both benchmarks, so the improvement mainly comes from the three-stage training rather than from asking for help. Sibyl (Qwen3-0.6B) hands 64.8% of its steps on WebShop to the cloud, yet its success rate (80.4%) is on par with GiGPO w/SFT (80.6%), which does not use the cloud at all at inference. Therefore, I find that for Sibyl (Qwen3-1.7B) with 3% call ratio, training is the key, and for Sibyl (Qwen3-0.6B) with 64.8% call ratio, calling for help brings no benefit. Hence, the additional benefit of asking the cloud for help is not real or at least not demonstrated. (ii) The setup of the routing baselines looks problematic. RouteLLM hands 85.5% of its steps on WebShop to the cloud, yet its success rate is only 22.8%, far below the 46.4% of using the cloud alone. This makes it look as if the cloud model performs even worse than the fine-tuned small model.
-
Training the proposed method requires fairly strong assumptions and costs. (i) The decisive-disagreement mining in Stage 2 requires restoring the environment to the exact state at a given step, replacing one step, and re-running the rest. This is only feasible in resettable simulated environments such as ALFWorld and WebShop, and is hard to achieve in real web or device environments. (ii) Training depends on many hyperparameters. The paper neither describes the tuning process nor shows the cost of tuning, including the learning rates of the three stages, , , , and .
-
The objective and the training algorithm are disconnected, and the mechanism for limiting cloud calls is unclear. (i) The paper formulates a cost-regularized objective (Eq. 4) with a trade-off coefficient , but never appears in the training algorithm, and the paper only states that tuning provides an implicit control over this trade-off. (ii) The reward contains no cost term. The consultation-specific advantage only compares the returns of calling and not calling at matched states, so it is unclear why this design limits cloud calls. (iii) Nothing in the method prevents excessive reliance on the cloud, e.g., Sibyl (Qwen3-0.6B) hands 64.8% of its steps on WebShop to the cloud.
Questions for authors
- In the experimental setup, the local models are only as small as 0.6B and 1.7B, while the cloud model is Qwen3.7-Max. Is this setup reasonable? Why are there no other combinations, such as larger local models (e.g., Qwen3-4B or Qwen3-8B), other model families, or different cloud models?
- After the three stages of training, the small model seems to have already learned how to solve these tasks, with a call ratio of only about 3% for the 1.7B model. If call_cloud is disabled at inference, what is the success rate of the small model? Is it possible that the small model achieves a higher success rate than the large model? What if the large model generates some reasoning or a plan once before the task starts and then feeds it to the small model?
- The ablation study stacks Stages 1, 2, and 3 sequentially. What happens if Stages 1 and 2 are skipped and Stage 3 is learned directly? The ablation study needs to be more comprehensive and in-depth.