- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SpatialOPSD uses a frozen model conditioned on successful coding-agent traces to train a student that reasons without tools. It adds repetition masking and an unlikelihood penalty, then evaluates spatial reasoning and transfer.
Critical strengths
The ViewSpatial gains are useful evidence of transfer beyond the training source. Results across model sizes and the privileged-context ablations also give the approach more support than a single benchmark result would. The practical goal is clear: use expensive agent traces during training while keeping inference free of tool calls. The experiments address that setting directly, and the frozen teacher makes the source of privileged context reasonably easy to follow.
Critical weaknesses
The leakage metric is my main concern. Appendix A.3 counts any use of “agent”, “trace”, or “tool”, including ordinary uses. A lower score could mean a change in vocabulary rather than less reliance on unavailable information. Table 3 also favors masking alone on accuracy, at 53.0% versus 51.9% for the full method. Unlikelihood alone has lower measured leakage. These variants make different tradeoffs, but the lexical metric makes those tradeoffs hard to interpret. A semantic check is needed before treating the lower keyword rate as a correction of the underlying failure.
The SFT baseline uses 4,459 tool-free rationales, while OPSD uses 5,008 agent-correct instances. The comparison changes the training examples and targets as well as the objective. This leaves open whether OPSD improves how the model learns from traces, or whether the benefit comes mainly from which traces it sees.
Questions for authors
Can you manually check a sample of keyword matches for actual unsupported trace references? How does SFT on the same summarized agent traces compare with OPSD? The matched SFT comparison would help distinguish the value of agent traces from the value of the OPSD objective.
4 comments
Dear Reviewers,
Please read the authors’ rebuttal and join the discussion. Kindly clarify whether your main concerns have been addressed and raise any follow-up questions promptly so the authors have time to respond.
Thank you for your time and effort!
Follow-up before the discussion deadline – Paper 7315
Dear Reviewer dRs3,
Thank you for your time and thoughtful consideration of our submission.
As the discussion period is drawing to a close, we would greatly appreciate your response at your earliest convenience, ideally before the deadline, so that we have an opportunity to address any remaining questions or concerns.
We understand that you have a busy schedule and sincerely appreciate your time and engagement.
Best regards, The authors
Follow-up on Our Responses
Dear Reviewer dRs3,
Thank you for your careful review and thoughtful feedback.
As the discussion deadline approaches, we would appreciate your feedback on whether our responses have addressed your concerns. If any issues remain unresolved, we would be happy to provide further clarification.
Thank you again for your time and consideration.
Best regards, The authors
Response to Reviewer dRs3
We deeply thank Reviewer dRs3 for the insightful review! We greatly appreciate the reviewer's recognition of our practical tool-free inference goal, clear experimental setup, and strong transfer results on ViewSpatial. Directly following Reviewer dRs3’s valuable suggestions, we conducted targeted experiments to fully resolve the two main concerns: (1) We implemented a manual audit and LLM-based semantic check as suggested, verifying that SpatialOPSD massively reduces genuine semantic reliance on unavailable traces (from 82.5% to 25.1%), not just keyword usage; and (2) We evaluated SFT and SpatialOPSD on the exact same 2,644 examples, conclusively showing that our gains (+1.6% average, +8.5% on ViewSpatial) come from the OPSD objective itself rather than data differences. Details are provided below.
A1: We thank the reviewer for this valuable suggestion. Following it, we added both a manual audit and a semantic leakage evaluation.
Manual audit. We manually inspected all 95 responses from SpatialOPSD on MindCube-Tiny that the keyword rule flags. 86 of them (90.5%) contain genuine references to unavailable privileged information, e.g., "The reasoning trace says: > The camera moved diagonally forward and right." The remaining 9 use the keywords in their ordinary sense, either as objects in the scene ("a black tool cabinet") or as the verb trace ("if you trace the line of sight").
Semantic evaluation. We further used Qwen3.7-plus to judge every response. A response counts as leaking if it refers to, quotes, or relies on any source that is absent from the model's input. This includes paraphrased references that contain none of the keywords (e.g., "the provided reasoning"). Ordinary uses of the keywords, quotations of the question, and image descriptions are excluded.
For vanilla OPSD, the keyword and semantic rates nearly coincide, which confirms that the baseline failure is genuine reliance on unavailable traces. Because the semantic criterion is stricter and also captures paraphrased references, it yields a higher absolute rate for SpatialOPSD. Even under this criterion, SpatialOPSD reduces leakage from 82.5% to 25.1%, while also improving accuracy on both benchmarks. This shows that the improvement reflects substantially less reliance on unavailable information rather than a change in vocabulary alone. We will add the judge prompt, and the manual audit to Appendix.
A2: We sincerely thank the reviewer for this insightful comment. Raising the question of data parity is crucial for a rigorous evaluation. Does SpatialOPSD’s advantage merely stem from data size or example selection? — No. A strictly controlled experiment on an identical dataset confirms its inherent algorithmic superiority.
To thoroughly address your concern regarding training data discrepancies (originally 4,459 for SFT vs. 5,008 for SpatialOPSD), we conducted an additional ablation study. We extracted the exact intersection of both datasets—yielding 2,644 shared examples—and trained Qwen3.5-9B from scratch using both methods under their respective optimization procedures. This strictly eliminates any confounding factors related to data volume or sample selection.
Key Takeaways from the Matched-Data Comparison:
Consistent Overall Advantage: Even when restricted to the exact same 2,644 examples, SpatialOPSD outperforms SFT on average (50.2% vs. 48.6%, +1.6%), driven by a massive +8.5% surge on ViewSpatial (46.5% vs. 38.0%).
Reproduced Qualitative Pattern: The results perfectly mirror our full-data findings—while SFT leans towards MindCube-Tiny (59.2% vs. 53.9%), SpatialOPSD dominates on ViewSpatial and secures a higher overall average.
Conclusion: By strictly locking the training data, this experiment decisively proves that SpatialOPSD’s superior performance is not an artifact of data scale or curation, but a direct result of its distinct supervision and optimization paradigm.