- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Selection-based Structured Reasoning (SSR), a framework that replaces freeform autoregressive reasoning in ReAct-style multimodal search agents with a selection step from a small, fixed library of six manually written natural-language reasoning candidates. SSR is evaluated on seven multimodal search benchmarks using Qwen3-VL at 2B and 4B scale, under four training objectives: GRPO, GSPO, SAPO, and SFT. The headline result is that the 4B SSR+GRPO model achieves 61.37% average success rate -- competitive with the best 4B baseline TAPO+GSPO at 61.25% (Table 1) -- while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28-54% (Table 2).
Critical strengths
- The over 90% reduction in per-turn reasoning latency and 28-54% reduction in per-question model latency (Table 2) are substantial and consistent across all five training configurations tested (GRPO, GSPO, SAPO, SFT at 4B; GRPO at 2B). The p95 reasoning latency of 0.071s versus 0.798-1.892s for baselines in Table 3 demonstrates not only speed but predictability, which is critical for the application.
- The paper tests SSR under four distinct training objectives (GRPO, GSPO, SAPO, SFT), providing evidence that the selection-based formulation is a general drop-in replacement rather than a method-specific trick. The comparison against four efficient-reasoning baselines in Table 3 is on the same GRPO pipeline with the same backbone, making it a controlled comparison.
Critical weaknesses
- The method is functionally action-type classification, not reasoning. The six candidates in Table 6 each correspond directly to one of the available tools or answer modes. The paper's own analysis reports 99.9% alignment between selected reasoning and expected action type (Section 4.6). Appendix G shows freeform reasoning does not improve over no reasoning at all (58.65% vs. 58.91% under GRPO), meaning the candidates provide no deliberative value beyond determining which tool to call next. Without a comparison against a simple action-type classification baseline (e.g., a 6-way linear head over the hidden state), it is impossible to determine whether the full parallel-scoring machinery over natural-language text adds value beyond classification.
- All seven benchmarks involve the same multimodal search task format with the same small tool set. The library of 6 candidates was hand-crafted specifically for this task family. There are no experiments or analyses on any other agent domain, where reasoning patterns are more diverse and less easily enumerable.
Questions for authors
- Given the 99.9% alignment between selected reasoning and action type (Section 4.6), have you compared against a simple 6-way classification head (e.g., linear layer over the last hidden state) that selects the action type and then injects the corresponding template? This would isolate whether parallel scoring of natural-language candidates contributes beyond classification.
- The Appendix G comparison showing freeform reasoning underperforms no-thinking (58.65% vs. 58.91%) is potentially the most important result in the paper, as it suggests SSR's gain comes from constraining action selection rather than enabling reasoning. Could you discuss this finding more prominently and reconcile it with the paper's reasoning framing?