acceptodds
Withdrawn from ICLR 2027

NAVIGATE: Next-Action Verifiable and Flexible Incentives with Guided Adaptive Tool Execution for Multimodal Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) remain brittle on tasks requiring multi‑step perception–reasoning and tool use. Existing approaches degrade perceptual accuracy, overfit to tool‑call patterns, or generalize poorly to multi‑step rollouts—manifesting as irrelevant or skipped tool invocations. We present NAVIGATE, a multi‑stage framework that teaches MLLMs to plan next actions, call perception specialists, and verify outcomes end‑to‑end. The first stage performs tool‑aware adaptive sampling: a curriculum that weighs trajectories by tool usage, compositionality, and perceptual hardness, progressing from easy grounding to complex multi‑tool reasoning. The second stage augments supervised fine‑tuning (SFT) with a planner head that forecasts action sequences and regresses to action embeddings to respect similarity among tools. The final stage applies reinforcement learning (RL) with verifiable (terminal accuracy, format) and flexible (step‑wise consistency, embedding‑transport discrepancy, variance) rewards to optimize interactive rollouts. On multiple benchmarks, NAVIGATE yields around 10% gains over SFT and RL baselines across different backbones. We also conduct a series of ablation experiments and analyses to gain deeper insight into our framework.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal reasoning, tool calling, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_coDzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

NAVIGATE is a three-stage framework for training multimodal models to answer visual questions through external tool use. It combines a tool-aware curriculum, supervised fine-tuning with an auxiliary head that predicts future tool actions, and reinforcement learning using final-answer correctness, format compliance, planner–execution consistency, MMD-based embedding alignment, and covariance-stability rewards. The paper reports improvements across six visual perception and reasoning benchmarks, multiple model backbones, and several component ablations.

Critical strengths
  1. The paper addresses a useful problem: learning when and how to compose external tools for multimodal reasoning. Combining curriculum learning, auxiliary action prediction, and interactive reinforcement learning provides a plausible training approach to this problem.
  2. The evaluation includes multiple backbones and a matched SFT-to-GRPO baseline. In Table 8, NAVIGATE improves the overall score from 48.9 with matched vanilla GRPO to 51.3, providing evidence that the proposed components collectively offer benefits beyond simply adding reinforcement learning.
Critical weaknesses
  1. The central reward design lacks sufficient justification, and the presentation makes the method difficult to assess. The paper does not adequately explain why MMD-based alignment between input and output embeddings should improve visual grounding, or why similar output covariance across successive steps should improve tool reasoning. Different stages of a valid tool chain may naturally require different representations, and these rewards do not directly measure correctness. The writing and organization further obscure the method: notation is introduced without sufficient explanation, the relationship between the planning head and actual tool execution remains unclear, and the overview figure does not illustrate how the proposed rewards are computed or influence training. The qualitative examples show successful trajectories but provide limited insight into these mechanisms. A clearer formulation and a worked example connecting planning, execution, and reward computation would substantially improve the presentation. These shortcomings affect understanding and reproducibility of the main contribution, beyond stylistic preferences.
  2. The experimental evidence does not sufficiently substantiate the claims about long-horizon, flexible tool orchestration. The six benchmarks are useful for measuring final-answer quality, but improvements on them do not by themselves establish better adaptive planning. Table 11 reports only 2.86 executed tool calls per example for the default model, while extending the prediction horizon from 5 to 50 improves the overall score by just 0.4 points, without reported uncertainty estimates. The held-out-tool evaluation is relevant, but its training exclusions and evaluation protocol are insufficiently specified. There is also an unexplained reporting ambiguity: Table 8’s “without Stage III” configuration and Table 10’s “format + accuracy only” RL configuration have identical scores despite apparently different training procedures. Together, these issues limit confidence in the attribution and scope of the reported gains.
Questions for authors
  1. What motivates the MMD and covariance-stability rewards specifically for tool-assisted reasoning? How are their embeddings computed, and what evidence shows that these rewards improve grounding or useful tool execution rather than merely encouraging representational similarity?
  2. How is the embedding planner decoded into discrete tool predictions, and does it directly influence execution at inference? Please also specify how terminal accuracy and intermediate rewards are aggregated and converted into GRPO advantages.
  3. Why do the “without Stage III” results in Table 8 exactly match the “format + accuracy only” results in Table 10? What training procedures and retained components distinguish these configurations?
  4. In the held-out-tool evaluation, is an entire tool family excluded from training or only a particular API? What are the evaluation sample sizes, and how are unseen tools incorporated into the planner’s action space?
Reviewer_cRPVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes NAVIGATE, a training framework for multimodal reasoning with tools. It combines curriculum sampling of LATTE trajectories, supervised training with an auxiliary action planner, and GRPO with multiple reward components. The approach is evaluated on six benchmarks with several model backbones and ablation studies.

Critical strengths
  1. Implementation effort. The paper presents a training framework that combines several components for multimodal tool use.
  2. Experimental coverage. The paper includes experiments on multiple benchmarks and ablation studies of its main components.
Critical weaknesses
  1. Limited methodological novelty and insufficiently justified design. The framework combines several established techniques, but its methodological advance over existing tool reasoning frameworks remains unclear. Its positioning also understates prior work on planning across multiple steps, particularly AdaReasoner. The additional consistency, MMD, and covariance rewards introduce substantial complexity without sufficiently establishing their connection to faithful tool use. Some explanations are also inconsistent with the method: the discussion on page 19 attributes degradation to softmax and cross entropy saturation, although the default planner uses cosine regression.

  2. Substantial unexplained discrepancies in baseline results. Table 1 reports Qwen2.5-VL-72B scores of 40.2 on MathVista and 45.9 on MMStar, whereas its official model card reports 74.8 on MathVista_MINI and 70.8 on MMStar. Differences in evaluation splits or settings may contribute to these gaps, but the paper does not sufficiently document or explain them. Given their magnitude, these discrepancies substantially weaken my confidence in the experimental comparisons. The evaluated checkpoints, benchmark splits, inference settings, and scoring procedures require clarification.

Questions for authors

See weaknesses.

Reviewer_A8MGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

NAVIGATE proposes a three-stage training pipeline that teaches the multimodal LLMs to plan for the best tool-use and verify for answers. It proposes an adaptive training curriculum that progressively increases difficulty of tasks, and a planning head that predicts upcoming tool calls, and uses GRPO as the RL method.

Critical strengths
  • Results show that NAVIGATE is better than other methods.
Critical weaknesses
  • There is little novelty in the paper, and I find the extensive, overcomplicated mathematical formulations in this paper to be unnecessary.
  • There lacks sufficient evidence to show NAVIGATE is more effective, because even when training data and rollouts budgets are matched, it would appear that NAVIGATE introduces additional training phases that would give it more training opportunities.
Questions for authors
  • Table 8 introduces a matched experiment comparing to the GRPO baseline, with the same training data, rollout budget, and optimization steps. However, NAVIGATE introduces additional computational steps, could the authors clarify the total GPU time and training FLOPs for NAVIGATE and its baselines? How much of the improvement over vanilla GRPO persists under equivalent end-to-end computational budgets?

  • Does NAVIGATE actually improve tool-selection reliability? In Figure 4(d) it reports that the proportion of categorized errors belonging to irrelevant tool usage increases from 28% for LATTE to 45% for NAVIGATE. It does not appear to me that there is significant improvement on this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.