acceptodds
Under review as a conference paper at ICLR 2027

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

Abstract

Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their likelihoods given the current context, without an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Agentic Search, Efficient Inference, Multimodal LLM Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yGTMOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Selection-based Structured Reasoning (SSR), a framework that replaces freeform autoregressive reasoning in ReAct-style multimodal search agents with a selection step from a small, fixed library of six manually written natural-language reasoning candidates. SSR is evaluated on seven multimodal search benchmarks using Qwen3-VL at 2B and 4B scale, under four training objectives: GRPO, GSPO, SAPO, and SFT. The headline result is that the 4B SSR+GRPO model achieves 61.37% average success rate -- competitive with the best 4B baseline TAPO+GSPO at 61.25% (Table 1) -- while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28-54% (Table 2).

Critical strengths
  1. The over 90% reduction in per-turn reasoning latency and 28-54% reduction in per-question model latency (Table 2) are substantial and consistent across all five training configurations tested (GRPO, GSPO, SAPO, SFT at 4B; GRPO at 2B). The p95 reasoning latency of 0.071s versus 0.798-1.892s for baselines in Table 3 demonstrates not only speed but predictability, which is critical for the application.
  2. The paper tests SSR under four distinct training objectives (GRPO, GSPO, SAPO, SFT), providing evidence that the selection-based formulation is a general drop-in replacement rather than a method-specific trick. The comparison against four efficient-reasoning baselines in Table 3 is on the same GRPO pipeline with the same backbone, making it a controlled comparison.
Critical weaknesses
  1. The method is functionally action-type classification, not reasoning. The six candidates in Table 6 each correspond directly to one of the available tools or answer modes. The paper's own analysis reports 99.9% alignment between selected reasoning and expected action type (Section 4.6). Appendix G shows freeform reasoning does not improve over no reasoning at all (58.65% vs. 58.91% under GRPO), meaning the candidates provide no deliberative value beyond determining which tool to call next. Without a comparison against a simple action-type classification baseline (e.g., a 6-way linear head over the hidden state), it is impossible to determine whether the full parallel-scoring machinery over natural-language text adds value beyond classification.
  2. All seven benchmarks involve the same multimodal search task format with the same small tool set. The library of 6 candidates was hand-crafted specifically for this task family. There are no experiments or analyses on any other agent domain, where reasoning patterns are more diverse and less easily enumerable.
Questions for authors
  1. Given the 99.9% alignment between selected reasoning and action type (Section 4.6), have you compared against a simple 6-way classification head (e.g., linear layer over the last hidden state) that selects the action type and then injects the corresponding template? This would isolate whether parallel scoring of natural-language candidates contributes beyond classification.
  2. The Appendix G comparison showing freeform reasoning underperforms no-thinking (58.65% vs. 58.91%) is potentially the most important result in the paper, as it suggests SSR's gain comes from constraining action selection rather than enabling reasoning. Could you discuss this finding more prominently and reconcile it with the paper's reasoning framing?
Reviewer_xexqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Selection-based Structured Reasoning (SSR), which replaces free-form reasoning in small multimodal search agents with selection from a fixed library of reusable natural-language candidates. At each interaction turn, the model scores the candidates from the current context, selects one, inserts it into the context, and then generates the corresponding tool call or final answer. Because the candidates are known in advance, their token likelihoods can be computed in parallel, substantially reducing reasoning latency. Experiments on seven multimodal search benchmarks with 2B and 4B Qwen3-VL models show competitive success rates and large reductions in model-inference latency across several training objectives.

Critical strengths
  1. The method design is reasonable for the stated problem. SSR replaces costly free-form reasoning with a small set of reusable reasoning candidates while retaining flexible, context-dependent action generation. The candidate-selection mechanism is integrated into the language model and does not require an additional classification head.
  2. The experiments provide broad coverage. The paper evaluates multiple multimodal search benchmarks, model sizes, training objectives, and reasoning baselines, and includes ablations on the candidate representation, decoding strategy, and library size. This gives reasonable support for the generality of the reported efficiency gains.
Critical weaknesses
  1. The performance comparison is not fully controlled. SSR receives an additional format reward for matching each selected reasoning candidate with its expected tool, while free-form reasoning does not. This couples candidate selection with tool supervision, so the reported success-rate gains cannot be attributed to the SSR decoding mechanism alone.
  2. The efficiency claim is evaluated too narrowly. The reasoning library is manually designed and fixed, and latency measurements exclude tool execution, retrieval, and judging. The results therefore support faster model inference in the tested setting, but do not establish general end-to-end efficiency gains.
Questions for authors
  1. Under exactly the same accuracy and format rewards, without the additional candidate-tool matching reward for SSR, what are the success rates and tool-use statistics of GRPO-freeform and GRPO-SSR?
  2. How is the “expected tool” associated with each candidate defined? Is it specified manually, derived from annotations, or inferred from trajectories?
  3. What are the end-to-end wall-clock times, including image search, text retrieval, cropping, and LLM judging?
  4. Do the gains persist on a candidate library written independently of the training benchmarks or on unseen benchmarks? Could you report results over multiple random seeds to separate candidate-library adaptation from the general selection mechanism?
Reviewer_63sNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Selection-based Structured Reasoning (SSR) for small multimodal search agents. Instead of generating freeform reasoning at each turn, the acting language model scores reusable natural-language candidates using length-normalized conditional likelihoods, selects one, and generates a context-specific action. Teacher-forced scoring parallelizes candidate evaluation while sharing the history KV cache. The selector and action policy are trained with SFT or reinforcement learning; the practical RL implementation uses a detached-competitor approximation.

Experiments cover seven benchmarks with Qwen3-VL 2B/4B models. For 4B GRPO, average success rises from 58.65% to 61.37%, while mean reasoning latency falls from 0.896s to 0.061s and model latency per question from 5.456s to 2.514s. The latency measurements explicitly exclude tool execution.

Critical strengths
  1. A practical representation change with substantial measured efficiency gains. Replacing autoregressive reasoning generation with batched likelihood scoring addresses a concrete inference bottleneck without adding a separate classifier. Table 2 reports reasoning-latency reductions above 90% across all five objective/model-size configurations, with question-level model-latency reductions of 28–54%. Table 3 also reports favorable mean and p95 latency against efficient-reasoning baselines. The consistency of the efficiency benefit is the strongest evidence supporting acceptance.

  2. The evaluation provides useful evidence beyond comparison with freeform reasoning alone. The paper includes no-thinking comparisons, full-text versus index-only selection, autoregressive structured reasoning, and library-size ablations. SSR exceeds no thinking on all seven benchmarks in Appendix G, and full-text selection exceeds index-only selection in Table 4. These results support the usefulness of context-dependent selection among structured candidates in the studied search setting.

Critical weaknesses
  1. Differences in reward design complicate attribution of the accuracy gains. Appendix F states that SSR receives full format reward only when the generated tool matches the expected tool of its selected candidate, whereas the other conditions have different format requirements. Since the library closely corresponds to tool-selection decisions, this additional consistency supervision could contribute to the accuracy differences. The index-only comparison controls for this within SSR, but does not isolate its contribution relative to freeform or no-thinking training. This limits the strength of the claim that the selection mechanism itself improves accuracy, although it does not invalidate the measured inference-speed benefit.

  2. The effect of the practical selector-gradient approximation remains empirically unresolved. Appendix B clearly distinguishes the exact categorical gradient from the approximation that differentiates only the selected score and retains rollout-time competitor scores. Table 2 shows lower success under SSR for GSPO and SAPO, which the authors hypothesize is related to this approximation during repeated updates. However, no direct comparison establishes that explanation or quantifies its effect. This leaves uncertainty about how reliably the proposed training formulation transfers across optimization settings.

Questions for authors
  1. Can you isolate the effect of candidate–tool consistency rewards from that of reasoning selection? A reward-matched comparison, or an ablation removing the additional consistency requirement from SSR, would help establish how much of the accuracy improvement remains.

  2. Can you quantify the effect of detached competitor scores, preferably through a small-scale exact-gradient comparison and/or varying the number of updates per rollout? In particular, does this account for the GSPO/SAPO performance decreases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.