acceptodds
Withdrawn from ICLR 2027

UniNavManip: Graph-Constrained Belief- Space Control for Unified Mobile Navigation and Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-horizon mobile manipulation must coordinate task progress, visual memory, and heterogeneous base–arm control under partial observability. Existing visuomotor policies often model motion, task structure, or visual history separately, leaving mode transitions and verified progress weakly coupled.We present UNINAVMANIP, a vision-language-action controller that combinesa language grounded TaskGraph,typed head–wrist memory, mode-routed hybrid actions,and receding horizon candidate selection.The graph restricts proposals using verified predecessors and confirmed preconditions; the controller executes short prefixes and updates the graph only after fresh-observation verification. In simulation-only physics-based IsaacSim evaluations on 240 held-out episodes from three scenes, UNINAVMANIP reaches 61.7% complete-task success,compared with 57.1% for the strongest adapted public baseline and 49.6% for a matched internal hierarchical baseline. It also records the lowest collision rate at 0.15 per episode.

Primary area:
applications to robotics, autonomy, planning
Keywords:
loco-manipulation, vision-language-action models, task graphs, belief-space control, long-horizon robot control

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_DNX9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes UniNavManip, a unified framework for long-horizon mobile navigation and manipulation. The method combines a language-grounded TaskGraph, dual-view visual memory, mode-specific action heads, and a learned world-action model for candidate selection. TaskGraph constraints restrict subgoal and mode selection, while observation-based verification updates task progress. The world-action model evaluates candidate trajectories based on predicted progress, collision risk, and uncertainty. Experiments in Isaac Sim show improvements in task success rates and collision reduction compared with existing baselines. The paper also includes component ablations, robustness tests, and inference efficiency analysis.

Critical strengths
  1. A comprehensive framework for mobile manipulation.

The paper addresses important challenges in mobile manipulation, including task dependencies, visual memory, base-arm coordination, and action selection. These components are integrated into a structured control framework. This is a meaningful system-level contribution and involves considerable engineering effort.

  1. Positive experimental results with useful analysis.

The proposed method improves task success rates and reduces collisions compared with the evaluated baselines. The component ablations, robustness tests, and efficiency measurements provide useful evidence of the framework's effectiveness.

Critical weaknesses
  1. The core mode-transition mechanism is insufficiently explained and evaluated.

The framework uses separate action heads for NAVIGATE, ALIGN, and MANIPULATE. Therefore, accurate mode transitions are essential, since an incorrect mode may prevent the robot from executing the actions needed to complete the task.

However, the paper does not clearly explain how interaction-region, reachability, and alignment conditions are computed and verified. The supplementary material also lacks sufficient details about the physical feasibility checks.

More importantly, the paper does not directly evaluate the accuracy and robustness of mode transitions. Incorrect transitions may lead to premature manipulation, repeated alignment, or stalled execution. Although a RECOVER mode is included, its ability to handle transition errors is not adequately demonstrated.

The adaptability of these conditions is also unclear. Different environments and robot embodiments may have different workspace constraints, reachability conditions, and motion capabilities. The current evaluation does not sufficiently demonstrate whether the proposed transition mechanism can generalize across such settings.

  1. The contributions of the learned world model and external physical filtering are not sufficiently distinguished.

The framework generates candidate trajectories using an expert-trained policy and filters them using physical feasibility checks before applying the learned world-action model. Therefore, the quality of the candidates and the external filtering mechanism may already contribute significantly to the final performance.

Although the ablation study shows that world-action rollout improves task success, it does not fully establish how much of the improvement comes from learned future prediction rather than high-quality candidates or external physical checks.

In particular, the implementation of the physical feasibility filter is not sufficiently explained. It is unclear whether the online checks rely only on estimated robot observations or also use simulator ground-truth information.

Furthermore, the paper does not directly evaluate the prediction accuracy of the learned dynamics and collision models or their candidate-ranking ability. Comparisons with simpler selection methods are also limited. As a result, the necessity and independent contribution of the learned world model remain unclear.

Questions for authors

Clarification Questions

1.Mode transitions: How exactly are the interaction-region, reachability, and alignment conditions computed? Are these conditions learned or rule-based? How does the framework detect and recover from incorrect mode transitions?

2.Physical feasibility checks: How is the physical feasibility mask computed during online inference? Does it rely only on robot observations and estimated states, or does it use any simulator ground-truth information? Please clarify the exact inputs and procedures.

Additional Experimental Evidence

3.Mode-transition robustness and generalization: Could the authors directly evaluate mode-transition accuracy and failure cases under different object positions, workspace constraints, and scene layouts? How much of the transition mechanism can be transferred to a different robot embodiment without redesign or retraining?

4.World-model contribution: Could the authors evaluate candidate-ranking accuracy and compare the learned world model with simpler selection methods, such as policy-prior ranking, geometric collision filtering, or a direct action-conditioned value model, using the same candidate trajectories? This would help separate the contributions of candidate generation, physical filtering, and learned future prediction.

5.Candidate quality and prediction accuracy: Could the authors report the quality and diversity of the generated candidates, as well as the prediction accuracy of the dynamics and collision models? In particular, how often does the candidate set contain a feasible or successful trajectory before world-model ranking?

Reviewer_eCA5OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UniNavManip targets long-horizon mobile manipulation. It combines a language-based TaskGraph, visual memory from head and wrist cameras, a flow-matching controller, and an ensemble-based method for selecting actions. During execution, it only runs the first part of a plan, checks the result using new observations, and updates the TaskGraph before proceeding. In Isaac Sim, the method achieves a 61.7% success rate over 240 trials in three unseen scenarios. The adapted SERF-VLA baseline achieves 57.1%, while a hierarchical baseline reaches 49.6%. UniNavManip also reports an average of 0.15 collisions per task. The paper includes ablations, perturbation tests, and an analysis of inference cost.

Critical strengths
  1. The method distinguishes between confirmed facts and tentative facts. This design helps the system track which task states have actually been verified and which are still tentative. This preventing the system from relying on incorrect facts.

  2. The TaskGraph provides a common interface for planning and execution. This is useful for navigation, alignment, and manipulation in partially observable environments.

  3. The ablation studies show the effectiveness of the system, and the RECOVER module also effectively improves the overall success rate.

Critical weaknesses
  1. The overall experimental evaluations is somewhat limited and lacks experiments in real world. The method was tested only in three unseen simulated scenarios. It remains unclear whether the performance gains reported in the paper generalize to other scenarios, task configurations, or real-world physical environments.

  2. The selection process for candidate solutions requires a more direct and clear explanation. Further elaboration is needed regarding how candidates are generated and how they incrementally satisfy graph constraints.

  3. The novelty of this work needs to be articulated more clearly. Existing graph-based approaches, such as SayPlan (Rana et al.), VeriGraph (Ekpo et al.), RePlan (Skreta et al.), EPoG (Yang et al.), and MoMaStage (Li et al.), have already explored similar concepts within the realms of planning, verification, or closed-loop execution. The paper should more explicitly state the technical innovations of UniNavManip and provide direct experimental evidence to substantiate these contributions.

Questions for authors
  1. How well does this method generalize across different scenarios and task configurations? Are there any real world experimental results or evidence suggesting that the reported performance gains can transfer to physical environments?

  2. During the planning process, how does the method ensure that candidate plans satisfy the constraints of an evolving TaskGprah? Furthermore, how does the system verify the feasibility of a plan during execution as the task state changes?

  3. What are the key technical differences between UniNavManip and existing graph-based methods? Which specific mechanisms can be validated through comparative experiments?

Reviewer_aYhNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes UniNavManip for long-horizon mobile navigation and manipulation, using a task graph to track progress and constrain action selection. The method combines typed visual memory, navigation and manipulation policies, and belief-space rollouts to select candidate plans. Evaluation is conducted in Isaac Sim, with comparisons against existing methods, component ablations and perturbation tests.

Critical strengths
  1. The motivation is clear. Maintaining consistent task progress across navigation, manipulation and recovery is a useful problem.
  2. Table 2 shows sizable performance drops when graph projection, coupled ALIGN or rollouts are removed. These results support the usefulness of these components in the tested setting.
  3. The supplement provides useful implementation and training details. Matching the memory capacity and candidate budgets across relevant ablations makes the comparisons more informative.
Critical weaknesses
  1. The system makes completion decisions irreversible while relying on a learned verifier. The paper does not report false-completion rates or explain how a mistaken completion is corrected, even though completed nodes cannot be revised and determine which subsequent actions are allowed.

  2. The rollout ablation does not isolate the benefit of multi-step prediction. The no-rollout variant selects by policy prior, removing progress and risk scoring at the same time. It is unclear whether the gain requires a world model or could be obtained by directly scoring the same candidates.

  3. The evaluation mainly tests transfer to new scenes using the same eight task templates. The paper does not do a great job of showing when graph revision is actually needed, such as changed prerequisites or an incorrect initial plan. The no-revision variant already achieves 59.2% compared with 61.7% for the full model.

  4. “Recovery SR” only requires a feasible action after RECOVER, rather than successful completion of the failed subgoal. The overall Task SR ablation does show that RECOVER helps, but this metric gives limited information about what was actually recovered. Table 3 also lacks comparisons with other methods under the same perturbations.

  5. There are no real-robot experiments. For a system whose main contribution includes base–arm coordination, visual verification and recovery, I would expect some evidence that the complete loop works with real sensing and execution errors.

Questions for authors
  1. What happens when the verifier incorrectly marks a subgoal as complete? How can the system correct this while preserving completed nodes?

  2. How does the six-step rollout compare with a direct progress/risk scorer over the same four candidates? Does increasing the rollout horizon actually improve action selection?

  3. On which tasks does graph revision help over the no-revision variant? Can you show cases with changed prerequisites or incorrect initial plans, rather than only different scenes?

  4. After RECOVER, how often is the failed subgoal actually completed? How do SERF and π0.5 perform under the same perturbations?

  5. Are there any real-robot results for the complete system, including verification and recovery?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.