- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Spatial Harness, which aims to improve the reliability of tool-assisted spatial reasoning. At its core is a closed evidence-acquisition loop composed of a planner, an executor, and a verifier: the planner schedules primary and backup routes for missing evidence, the executor generates programs that invoke visual tools, and the verifier applies semantic and spatial checks to the returned results, writing them into committed memory or unresolved memory accordingly. When no evidence is retained, the system executes backup routes; when evidence is still missing, it replans based on the updated memory. The method is evaluated on multiple benchmarks.
Critical strengths
-
The research is innovative in its focus on the problem that visual tools may "return plausible but incorrect results." Distinguishing tool outputs from trustworthy evidence helps prevent perceptual errors from being amplified by downstream reasoning, which is of practical significance.
-
Verification results determine how evidence enters memory and influence the triggering of backup execution and replanning. This makes verification an explicit execution mechanism and clarifies the responsibilities and training objectives of each role.
-
The paper establishes a mechanism for actively seeking alternative evidence after errors are detected and integrates it into the method, enabling more flexible evidence accumulation across viewpoints and across steps.
Critical weaknesses
- I think the paper already has sufficient novelty, but the introduction still frequently overclaims its contributions. The paper's innovation lies in verification-controlled memory writes, evidence states, and recovery rules. However, existing methods (e.g., SpatialClaw[1]) already include strategies for checking tool outputs and cross-validation, so the incremental novelty at the base-method level needs to be delineated more accurately, and the related text should be revised accordingly.
[1] SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning (arXiv:2606.13673).
-
Trying tools offline to form a reference ordering, and using this ordering to determine primary and backup routes, may cause misunderstanding and errors in OOM environments, and it inherently restricts generalization to environments with different tool sets.
-
Unresolved memory also participates in the evidence-coverage determination and can terminate the backup-route search of the current round. However, retaining a candidate does not imply that the ambiguity affecting the answer has been resolved. The current description does not sufficiently explain when the system will continue disambiguation and when it considers ambiguous evidence adequate to support the final answer.
-
I do not consider the training pipeline and algorithm to be a distinctive contribution: role-specialized teachers, DAPO, and MOPD are all well-established techniques, and the authors merely adopt them without introducing innovations on top.
Questions for authors
-
The paper reports inference-cost comparisons, but the planning and recovery strategies do not explicitly account for tool costs, latency, or the expected benefit of additional evidence.
-
Could this framework, conversely, be used to distinguish the reliability and importance of different tools? Going further, could tools be pruned to improve accuracy and reduce cost?
-
Modeling visual tools as stochastic observation channels and interpreting evidence acquisition as an output-feedback process suggests that more systematic and quantitative control could be achieved based on observation-probability estimation, budgets, and related factors.
5 comments
Appreciation for the Encouraging Feedbacks and Summary of Revisions and New Experiments (2 of 4)
W2: Discussion about reference ordering on offline scenarios and newly arrived tools.
We thank the reviewer for opening a discussion about the tool reference ordering. The accuracy of Spatial Harness is highly robust to the tool ordering and can transfer well when new environments or new tools arrive.
1. For new environments. Our experiment results on Ego3D dataset have well illustrated the generalization capabilities to new enviroments. The offline tool accuracies are collected at our training set, which does not contain outdoor road environments, as featured by Ego3D datasets. However, Spatial Harness still performs overall the best among all methods listed, showcasing the strong robustness of our harness against environment changes.
2. For new tools. We made an extra ablation study to imitate the circumstances when new tools arrive. We randomly remove the measured accuracies for 50% of the tools in the functional tool sets, and set them as the average of the remaining tools. We measure the changes of their performances and costs at ReVSI datasets, as shown below.
| Method | ReVSI (Macro Avg.) | Avg. Num. of Tool Calls |
|---|---|---|
| Spatial Harness (GPT-5.2) | 67.4% | 9.37 |
| Tool Prior Randomly Removed (GPT-5.2) | 65.7% | 10.82 |
The results illustrate that some tools with no known accuracy priors does not heavily affect the overall performance and toool costs, since the tool itself is still capable of performing their tasks even without knowing its success rates, while the verifier is still capable of gating the tool errors, which is agnostic to tool ordering.
W3: About the unresolved memory.
In the verification process, each observation who has passed the semantic checks go into the memory. If all the passed observations can be merged to one entity in space, it goes into the verified memory. Otherwise, all distinct and unmergeable pieces go into the unresolved memory together, as the harness cannot judge which piece the question refers to. A typical usage of the unresolved memory is the counting task, as each piece is not negligible to perform the correct countings. We have also added a demo about appying unresolved memory to reasoning over moving objects in dynamic scenes in newly added Appendix G. , where unresolved memory serves as the register for locations of the moving entity at different timesteps.
When the memory acquision ends, all the unresolved memory, as well as the frames and questions are taken into the conclusion stage. The planning agent check both the memory and the question, and decides how to leverage the memory to make subsequent plans, or to reason over the memory and answer. The newly added qualitative examples in Appendix F provides some illustrative results (Figure 5, Figure 11 and Figure 13), where unresolved memory serves as the evidence of counting, or the conflicts to be resolved.
In all, the disambiguation of the unresolved memory happens at the concluding stage, and the usefulness of the unresolved memory depends on the task type and the actual config of the scene.
Appreciation for the Encouraging Feedbacks and Summary of Revisions and New Experiments (3 of 4)
W4: The innovation of the training strategy.
Instead of combining MOPD and DAPO, what we propose is a training signal gating mechanism named Mastery-guided Acquisition. The recipe answers an important questions the harness raises: what a compact model should take from teachers that are imperfect themselves due to dataset bias.
Why role teachers. Although the three roles share the context and solves the spatial reasoning tasks in common, their required capabilities differ. The planner reasons about meaning and missing evidence, the executor writes programs, and the verifier recognizes and matches objects across views. When one compact backbone learns all three at once from mixed rewards, these skills compete and the training collapses when the students try to learn these skills together. We therefore let each skill mature in its own teacher, then consolidate the teachers into one student with multi-teacher on-policy distillation (MOPD), the established way to merge specialist capabilities into one model (e.g., MiMo-V2-Flash) without collapsing.
What a teacher should offer, and what it should not. A role teacher should offer the student its procedure, and nothing more. The prodedure includes which tool answers an evidence gap, how to set up the call, and how to tell when a return is unusable. This know-how is largely structural, and it is exactly what the student lacks when every rollout fails. There the task reward is too sparse to teach it, so only a teacher can do so. What a teacher should not offer is its spatial verdicts, whose correctness is defined by labels derived automatically from 3D annotations, noisy enough that ReVSI re-annotates VSI-Bench. A noisy label misleads the reward only when it is included, but a teacher that has fit the noise carries it into every rollout it guides. This also explains a result in the table below: deployed alone, the teachers fall below the untrained harness, yet distilled through the gate they lift the student well above it. Competence-gated guidance puts this into practice: the teacher speaks only where the student still lacks the procedure. In each rollout group:
- Every response fails: the student cannot yet work the harness here, so the teacher shows how, alongside task reward.
- Some succeed: the student already has the skill, so task reward refines it with the teacher off, and the teacher's bias stays out.
- All succeed: mastered, skipped.
Concurrent work (RetireOPD, arXiv:2609.20784) likewise finds that teacher guidance should give way as the student becomes competent. Our gate acts per rollout group, so each role keeps its teacher wherever it still fails. The training thus mirrors the framework: the harness admits a tool output only after the verifier accepts it, and the training admits teacher guidance only where the student needs it.
Evidence and contributions. To showcase the effectiveness of our confidence-gated acquisition mechanism, we perform another ablation where the teacher signal is always on during the MOPD process. Applying DAPO, MOPD or both to every group leaves the harness below its untrained level. Competence-gated guidance raises it, and is the only one to surpass all three teachers on held-out role contexts. The updated ablation study in the paper shows that simply assembling DAPO and MOPD together also hurts the training dynamics.
| Training | ReVSI (Macro Average) | Validation Accuracy (in ablation study) |
|---|---|---|
| None (untrained harness) | 63.7 | – |
| Separate role teachers | 61.9 | 84.3 |
| Multi-target DAPO | 51.0 | 74.1 |
| MOPD from role teachers | 52.3 | 74.1 |
| DAPO + MOPD, teacher always on (new) | 55.8 | 71.7 |
| Ours: DAPO + MOPD, competence-gated | 71.6 | 87.7 |
The framework alone lifts Qwen3-VL-8B from 49.1 to 63.7, and training adds 7.9 more. Our training contribution is therefore not DAPO or MOPD. It is the gate that lets a teacher transfer tool use without its bias. We have revised the introduction, Contribution 3, Section 3.3, and Fig. 4 accordingly.
Appreciation for the Encouraging Feedbacks and Summary of Revisions and New Experiments (1 of 4)
We sincerely thank the reviewer for the encouraging feedback and constructive suggestions. We particularly appreciate the recognition of our focus on silent failures of visual tools and the importance of distinguishing tool outputs from trustworthy evidence. Your comments have helped us clarify our contributions and motivated a more systematic analysis of tool reliability, marginal contribution, and acquisition cost, opening promising directions for more adaptive, resource-aware planning.
We have made substantial updates to the paper pdf and conducted new experiments to address all points:
Details are provided below. We are greateful to your insightful comments an look forward to any further feedback.
W1: About the contribution between Spatial Harness and SpatialClaw.
We thank the reivewer for mentioning the contribution comparison between Spatial Harness and SpatialClaw. We acknowledge that the tool result checking and cross reference have been established as prompts of SpatialClaw. However, a prompt to inspect the evidence is not equivalent to a verdict of its correctness, as prompts may not be faithfully executed by the agent. In Spatial Harness, we made the verification process mandatory and explicit by introducing a separate role and explicit verification stages on both the semantic intergrity and spatial consistency.
We further inspected the paper, code and running logs of both SpatialClaw and Spatial Harness, and discovered that a mandatory checking procedure is vital to guarantee the execution of the checking process. Spatialclaw assigned their checking process merely in the prompts, leading to the checking procedure becoming an occasionally triggered activity depending on the moment-to-moment capability of the agents capabilities. We used GPT-5.6-sol to analyze the agent trajectories of both methods. The table below shows the proportion of the checks actually fires when it is prompted to do so, and the proportion of the checks that finishes without errors.
From the table we can discover that prompting the check alone does not guarantee faithful checking behaviors, which lets the silent failures slip through, thereby showcasing the importance of the role-level mandatory checking procedures.
We have revised our Introduction Section to highlight the difference of verification mechanisms between Spatial Harness and SpatialClaw.