- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Spatial-JARVIS, a spatial regulator that operates independently of the interaction policy. Before an action is executed, it evaluates the proposed decision and provides action corrections and belief-revision feedback to guide current and subsequent decisions. The authors construct MetaSpatial-35K using simulator states, oracle trajectories, equivalent-action sets, and model-generated feedback, and train the regulator through SFT followed by MC-GRPO. They also introduce MetaSpatial-Bench for static evaluation. Experiments report improvements in monitoring and correction accuracy, an average task success rate gain of approximately 7.7 percentage points across six policies on SpatialWorld, and additional evaluations across environments and regulator sizes.
Critical strengths
Online improvements across multiple policies support the practical value of the approach. The evaluation goes beyond the authors’ static benchmark and includes six policies on SpatialWorld, together with transfer to EmbodiedBench-Nav. For example, Spatial-JARVIS improves Qwen3.5-27B’s success rate on SpatialWorld from 5.6% to 19.0%. These results provide meaningful evidence that the regulator complements the capabilities of the underlying policy.
The data construction and component analyses address relevant design choices. Equivalent-action sets avoid treating a single oracle action as the only valid decision, while policy-generated errors provide more realistic negative examples than random actions. Comparisons involving belief revision, general-purpose regulators, and a smaller regulator further support the value of specialized regulator training.
Critical weaknesses
The comparison with direct policy training does not fully isolate the benefit of the regulator architecture. The direct policy baseline primarily receives SFT, whereas Spatial-JARVIS receives both SFT and MC-GRPO and introduces additional model calls at inference time. Consequently, the comparison does not adequately separate the contributions of the regulatory structure, additional training, and additional inference resources. This limits the conclusion that training a regulator is more effective than directly improving the policy.
Supervision derived from privileged simulator states may not always be justified by the agent’s observations. Action validity and belief corrections are constructed using global simulator information, whereas the deployed regulator receives partial observations and interaction history. A reasonable exploratory action could be labeled incorrect because it does not advance the oracle’s current subgoal, and a correction could contain facts that cannot be inferred from the available observations. Scene and task separation addresses data overlap, but does not resolve this potential mismatch between the supervision and the information available to the regulator.
Questions for authors
Do you have a direct policy SFT+RL baseline using the same source data and comparable training compute, or a policy self-correction baseline with a matched inference-call budget? If so, do these comparisons still support the conclusion that training the regulator is more effective?
How do you ensure that negative actions are unreasonable given the agent’s observable information, rather than merely inconsistent with the globally informed reference trajectory? Have you audited whether belief-revision targets are inferable from the available observations and whether equivalent-action sets adequately cover reasonable exploratory actions?
Figure 1 appears to contain AI-generated visual artifacts. In the left panel, the top-down view and the perspective view appear spatially inconsistent; in the right panel, the cup appears to float rather than being held by the robot’s gripper. Could you clarify how this figure was produced? I recommend manually redrawing Figure 1 to ensure spatial consistency and physically plausible object interactions.
5 comments
W2 / Q2
We thank the reviewer for highlighting the distinction between factual correctness and support from local observations. We have considered this issue in our data construction.
For the "privileged information and local observations", we construct decision-level samples using subtask-specific rules. During Search, when the target has not been observed and no directional cues are available, we allow exploratory rotations in different directions without requiring the agent to follow the oracle’s particular search direction based on the target’s true location. During Navigation, the target location has already been observed in the interaction history. The simulator’s reachability graph is used to verify progress toward this known target while retaining multiple actions that advance navigation. Thus, our construction accommodates diverse exploratory choices when the target location is unknown and evaluates navigation toward previously observed targets, without requiring access to unobserved target locations.
For the “belief revision”, we provide Qwen3.5-Plus with both the ground-truth simulator state and the observations and interaction history available to the agent. These inputs serve distinct purposes: privileged states verify environmental facts and identify discrepancies between the policy’s beliefs and the actual state, while observations and interaction history provide the evidence for corrective feedback. The model is instructed to determine which beliefs require revision based on existing observations, action records, and execution feedback, rather than directly incorporating hidden simulator information into the corrections. For example, observations following a grasp attempt or explicit failure feedback can support the correction that an object has not been picked up. Conversely, a location that has neither been observed nor can be inferred from the history should not be directly disclosed.
As discussed above, our subtask-specific rules aim to retain reasonable alternatives as comprehensively as possible: Search accommodates non-redundant exploration in different directions, Navigation retains multiple actions that advance the goal, and Interaction and Completion generally retain only the reference action. We further manually reviewed 50 randomly sampled examples and found no omitted reasonable alternatives in 47 cases, yielding a coverage-check pass rate of 94%. These results provide direct empirical support for the coverage of our equivalent-action sets.
Q3
We clarify that Figure 1 was manually composed, with AI-generated assets used only for local robot illustrations and scene details that are difficult to draw by hand. These assets were intended to make the illustration more intuitive; however, limitations in AI generation led to inaccuracies in spatial relationships and physical details. We sincerely thank the reviewer for highlighting this issue and will manually redraw the figure in the revised manuscript to ensure its accuracy.
Thanks for your detailed responses, which partially solve my concern.
One part of Q2 remains open for me. The 47/50 audit checks whether reasonable actions were omitted. Did you separately check whether the factual claims in the belief-revision feedback could be supported by the agent’s observations and action history alone, with the simulator state hidden from the annotator? If so, could you report the sample size and result?
A minor clarification: my figure comment referred to Figure 1 in the original submission, showing the inconsistent views on the left and the floating cup on the right. Could you confirm that this is the figure you plan to redraw?
W1 / Q1
We thank the reviewer for the suggestions on comparison fairness. We address this concern through comparisons with direct policy training and inference-time self-correction.
We agree that the original comparison did not fully isolate the contribution of additional reinforcement learning. We therefore added a direct policy baseline trained with SFT followed by GRPO, initialized from Qwen3.5-9B and using data from the same source and at a comparable scale. This two-stage training directly optimizes the policy’s action generation. Together with the existing results in Table 4, the comparison is as follows:
All values are task success rates (%). Extending direct policy training to SFT + RL increases its overall success rate to 9.2%, while Spatial-JARVIS achieves 12.6%. This comparison provides further evidence for the effectiveness of training an independent regulator when both approaches receive two-stage training.
Moreover, we clarify that Table 7 already includes a policy self-correction baseline, termed Self-Reflection. Using the same Qwen3.5-27B interaction policy, this baseline reflects on its preceding decision at each interaction step. It achieves a task success rate of 8.2%, compared with 19.0% for Spatial-JARVIS, a difference of 10.8 percentage points. The additional output-token consumption is comparable: both methods generate approximately 200 additional output tokens per interaction step on average.
These comparisons support the benefits of Spatial-JARVIS over both direct policy optimization with SFT + RL and policy self-reflection with comparable additional output-token consumption.