- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This work observes that fine-tuning VLMs on embodied tasks leads to a degradation in certain spatial capabilities, and subsequently proposes a training recipe aimed at mitigating this loss.
Critical strengths
- The authors attempt to investigate an intriguing and seemingly counterintuitive phenomenon: why general vision-language capabilities are retained during embodied fine-tuning while spatial capabilities degrade.
- The paper is well-structured and clearly written, making it easy to follow.
Critical weaknesses
- Limited Methodological Novelty: The proposed "JOINT SPATIAL RE-AIMING" training strategy lacks substantial novelty. In essence, it appears to be nothing more than a conventional sample re-weighting scheme applied to the training dataset.
- Disconnected Observations & Lack of Mechanistic Explanation: The authors note that this degradation can transfer across domains, but this observation feels disjointed from uncovering the underlying mechanism. Crucially, the authors do not substantiate their core foundational motivation: why general vision-language capabilities remain intact while spatial capabilities specifically degrade.
- Artifact of Output Formatting Rather Than True Capability Loss: A closer examination of the provided error cases suggests that the purported "loss of spatial capability" is largely an artifact of output formatting mismatches (e.g., formatting discrepancies between point coordinates and bounding boxes ). In practical model training, such discrepancies typically stem from improperly curated fine-tuning data rather than any profound, mechanistic breakdown in spatial representation.
- Poor Presentation and Figure Quality: All figures in the paper appear to be AI-generated; they suffer from low information density, visual artifacts, and frequent text/element overlapping, which detracts from the professional presentation of the work.
Questions for authors
- What is the fundamental, principled difference between the proposed "JOINT SPATIAL RE-AIMING" and standard sample-level loss re-weighting strategies commonly applied to training datasets?
- Could the authors provide a rigorous explanation or empirical ablation elucidating the underlying cause behind why general vision-language capabilities are preserved while spatial reasoning abilities specifically undergo degradation?
5 comments
My problem remains unresolved
After reading the authors' response, most of it reads like AI-generated boilerplate. I'm maintaining my score for now: the points I'm concerned about could actually be stated in a sentence or two, and you don't need to produce a long block of text. Moreover, the results the authors report seem to support my original conclusion rather than overturn it. If they can answer my questions more concisely and accurately, I would be willing to raise my score to 2 (Q1).
Short Clarification on Q1 and Q2
For Q1: JSR uses standard LoRA fine-tuning, and its sampling can be expressed as re-weighting. Our contribution is identifying and repairing unexpected spatial localization degradation during large-scale MoE VLM post-training, rather than introducing a new optimization method.
We achieve substantial recovery with just 384 examples and 32 updates. Training Base directly on those examples improves localization but substantially lowers general-domain benchmark scores, so it does not meet our original goal of improving spatial abilities while retaining broader performance.
For Q2: Formatting alone does not explain the loss: on the same 1,861 examples where both checkpoints return one parseable point, precision drops from 33.37 to 27.19.
We would be happy to provide further details or discuss any other questions you may have.
Empirical Contribution and Localization Diagnostics
Thank you for raising the distinction between methodological novelty, the explanation of degradation, and its practical significance. We clarify the empirical contribution and the controls that distinguish response-format effects from geometric errors.
1. What differs from standard sample re-weighting?
JSR is situated in a three-stage workflow: pretrained MoE VLM → embodied adaptation → targeted spatial repair. Embodied adaptation improves several reasoning and understanding benchmarks while degrading some localization requirements. JSR starts from this adapted checkpoint and constructs a compact corrective dataset around the affected tasks and response requirements, with the objective of recovering localization while retaining adaptation gains. The adapted weights remain fixed while a fresh LoRA adapter learns from 384 examples over 32 updates; the adapter is then merged for inference.
Within this workflow, joint coverage specifies what must be supervised: correct spatial targets together with the required output types, target counts, and class-bearing responses across the repair tasks. Sample weights specify the relative emphasis assigned to those records. JSR uses standard supervised optimization, and its sampling mixture can be implemented through weighting once the records are fixed. Its contribution lies in the diagnosis-guided repair design and the empirical evidence for joint coverage.
The coverage controls show why this design matters: point-only repair reaches 46.03 Point F1 but only 0.02 class-sensitive box F1; box-only repair reaches 58.94 class-sensitive box F1 but leaves Point F1 at 16.26. Joint repair reaches 45.00 and 59.17, respectively. The new content-matched control also achieves strong repair with a simpler single-format variant (45.45 Point F1; 92.51 RefCOCO accuracy), supporting flexibility in serialization while retaining joint task coverage.
2. Why do localization and reasoning change differently?
In the original Qwen3.5 experiment, spatial reasoning / embodied understanding improves from 62.86 to 66.72 on average, while Point F1 falls from 26.98 to 12.32. Within localization, RefCOCO improves from 60.59 to 93.75 while multi-object geometry falls from 68.23 to 14.45. This selectivity motivates examining response requirements and geometric accuracy separately.
Our empirical results identify two components of the localization decline: increased sensitivity to response instructions and reduced coordinate precision even at a fixed output count. These task-specific requirements explain how localization scores can decline alongside improvements in reasoning or answer selection. The controls below quantify both components.
The weight interventions connect transfer to this diagnosis: subtracting the selected adaptation update recovers 51.5% of the loss from the pre-decline checkpoint to the adapted endpoint; adding the same update to a healthy checkpoint from an independent run induces a decline equivalent to 92.4% of the original decline across the selected update interval. Thus, transfer tests whether an adaptation update can induce localization degradation in another run, complementing the recovery test in the source run.
3. Is the degradation only a formatting artifact?
Formatting is an important part of the finding, but it does not explain all of the measured decline:
The last comparison holds parsing success and output count fixed: the precision decline is 6.18 points, with a paired 95% interval of [4.09, 8.32]. It therefore identifies geometric deterioration beyond invalid formatting or a point/box type mismatch.
4. Figure readability and provenance.
We acknowledge the readability concerns. The submission discloses AI assistance with implementation and presentation, including figures. To make the central coverage comparison directly inspectable within this response, we give the numerical results from the submitted Table 3:
The tables here give the numerical evidence for task coverage, response effects, and residual geometric decline directly. If accepted, we will revise the figures and tables throughout the paper for the camera-ready version to improve readability, including clearer labels, removal of overlapping elements, and more informative layouts.