- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes SOME-Flow, a utility-guided framework for query-adaptive structural agentic inference. It combines offline topology discovery, cost-aware topology matching, and selective post-execution evolution to balance accuracy and inference cost.
The authors evaluate SOME-Flow on five reasoning and code-generation benchmarks. Using Qwen3-8B as the primary executor, SOME achieves an average accuracy of 82.57%, outperforming the strongest automated baseline by 2.76 percentage points. The paper also includes ablation studies, accuracy–cost analyses, and cross-model transfer experiments.
Critical strengths
A useful perspective on utility-aware structural inference. The paper extends query-adaptive topology selection by explicitly considering whether additional computation is beneficial after initial execution. This provides a reasonable formulation for balancing performance and inference cost across different execution stages.
Systematic evaluation of the proposed components. The paper includes component-level ablations, accuracy–cost analyses, and cross-model transfer experiments. These experiments provide useful evidence for understanding the respective contributions of initial topology matching and selective continuation.
Critical weaknesses
-
Limited evaluation on genuinely agentic tasks. All five benchmarks primarily involve single-question answering, mathematical reasoning, or function-level code generation. None evaluates long-horizon planning, external tool interactions, or dynamic environmental feedback. While these benchmarks demonstrate the effectiveness of structured multi-LLM reasoning, they provide limited evidence for the broader claims about adaptive agentic inference. This is a significant limitation, as the proposed structural adaptation may be particularly relevant in more complex, multi-step agentic scenarios.
-
Insufficient evidence that the additional framework complexity is necessary. SOME combines offline topology search, a learned matcher, continuation prediction, and answer resolution, yet lacks comparisons against simpler alternatives, such as cost-aware routing with adaptive verification and answer resolution. Notably, full SOME improves over SOME-G1 by only 1.16 percentage points, while increasing relative inference cost from 11.04× to 19.43× (Table 1). Moreover, Table 2 lacks comparable inference costs for competing methods across executor models, making it difficult to assess the cross-model efficiency gains. Without stronger evidence against simpler, cost-matched baselines, it remains unclear whether the additional mechanisms are justified by their practical benefits.
Questions for authors
-
Could the authors evaluate SOME on at least one more realistic agentic benchmark, such as GAIA or WebArena, involving multi-step tool use or environmental interaction? This would help establish whether the claimed benefits of structural adaptation extend beyond conventional reasoning benchmarks.
-
How does SOME compare against a simpler baseline combining cost-aware routing with adaptive verification and answer resolution, under comparable inference budgets? In addition, could the authors report the token/API costs of competing methods in Table 2 and estimate the offline topology discovery and training costs?
-
How well does the learned routing and continuation policy adapt to stronger executor models? Since Table 2 transfers the policy trained on Qwen3-8B without recalibration, it remains unclear whether the same structural decisions remain cost-effective as the underlying model becomes more capable.