acceptodds
Under review as a conference paper at ICLR 2027

Same Reward, Different Ability: A Single-Variable Intervention Shows Task Formulation Governs What Spatial RLVR Learns

Abstract

Reinforcement learning with verifiable rewards (RLVR) is the standard recipe for improving spatial reasoning in vision–language models, and a growing diagnostic literature shows that such models often answer without looking. We ask the next question: what happens if the shortcut is removed before training? Nearly all point-level spatial benchmarks name the queried location as text coordinates, which the ground-plane relation makes informative about depth — a ridge regression that never sees the image beats a 7B vision–language model (VLM) on absolute depth. Holding the model, algorithm, reward, data volume, step count and the geometric distribution of training samples fixed, we vary only how the question refers to a pixel: drawn markers instead of coordinates. Across three paired seeds per arm, the marker formulation more than doubles metric accuracy and lifts every depth bucket off zero, while on these backbones the coordinate formulation leaves the hardest bucket at exactly zero under both RLVR and supervised fine-tuning. A decomposition arm shows the effect is the sum of two sub-interventions rather than one. The learned ability survives an adversarial set of off-ground points where the shortcut is void by construction, but does not transfer indoors, and ordinal accuracy stays below chance on a matched adversarial set — training removes the metric shortcut, not the ordinal position prior. Repeating the audit on two further backbones, including one from a different vendor, reproduces the diagnosis. Repeating the intervention shows that backbone capacity decides whether the formulation governs existence or merely degree — and that on the second vendor's model the effect does not appear at all, because that backbone learns no measurable metric ability here to begin with.

Primary area:
reinforcement learning
Keywords:
spatial reasoning, RLVR, vision-language models, shortcut learning, causal intervention, benchmark design

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dwyYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

RLVR has become a standard approach to improve the spatial reasoning capabilities of VLMs. However, current research indicates that VLMs often provide answers directly without observing the image. The authors attempted to examine this shortcut by controlling variables, specifically by removing text coordinates that contain depth information. They proposed three methods: using only image labels, using both labels and coordinates, and using only coordinates. Under RLVR training, all experiments using text coordinates outperformed the pure text model, while all experiments using visual labels outperformed their corresponding pure text models. This approach forces the VLM to examine the image, resulting in a non-zero performance in the case of the 7B model on worst-case bucket accuracy, which is a common phenomenon in other baseline methods.

Critical strengths
  1. The authors accurately located this kind of VLM shortcut problem after the perception process and reproduced it. A small probe model is able to decode depth information from the frozen representation, achieving an accuracy 2.2 times higher than the model's actual output accuracy. This means that the relevant information already exists in the model representation but is not reflected in the output.

  2. The authors find that a single comprehensive score is insufficient to distinguish whether the model has learned the geometric relationships or just matched the answer's distribution. In the experiments that trained on text coordinate data, the pass@1 score reached 0.1888, however, the worst bucket accuracy is still 0.

  3. Especially in depth estimation, to test whether depth information could be recovered from frozen visual representations, the authors trained a small probe. They found that the MLP probe achieved an accuracy of 0.3475 on the base model, compared to 0.1650 on the baseline. In contrast, the linear probe only provided an improvement of 0.075 to 0.085, indicating that relevant information already exists in the visual representation but cannot be directly obtained through a simple linear mapping.

Critical weaknesses
  1. The high-contrast visual markers may allow the model to read their row position in the image and apply the same planar geometry exposed by the text coordinates, resulting in a similar shortcut problem that the author claimed.

  2. Even without text coordinates, could the reward function itself cause the model to rely on the prediction of the median prediction? The tolerance band verifier may reward safe outputs.

  3. It remains unknown which computational processes are changed by the marker training. The authors demonstrate that the model's visual representation already contains depth information, but the performance improvement bring about by the proposed visual label training may also come from the model's implicit coordinate understanding of other visual cues rather than geometric representations.

Questions for authors
  1. The method proposed by the authors cannot be reproduced at all on the InternVL3.5-8B backbone network. Is this because of the different visual tower architecture? Does this indicate that the effectiveness of the proposed method is highly dependent on the architecture of the model itself?

  2. For the applicability of the method, the core mechanism proposed in the paper is specific to scenes with a dominant ground plane and may not be applicable to aerial images. Is it still effective for these scenes?

1 comment

Thank you. Each point led to a new experiment.

Does the marker just expose its row? (W1). Tested three ways (general response, part 2; App. Q Table 9). Grey image with the marker kept in place leaves exactly the information you describe; the marker models fall below the blind baseline (RLVR 0.254 → 0.089; SFT 0.291 → 0.043). A 180° rotation reverses the row cue; the marker models keep ranking by LiDAR depth (ρ = .39, .60) and anti-correlate with the row formula (every seed). On off-ground points the submitted marker RLVR model follows LiDAR depth rather than the formula (regression weights 0.38 vs 0.19). The same controls are reported for the mkc arm: on a grey image the SFT mkc arm still orders answers by the stated row (ρ = .39), as the coordinate arm does (.51).

Prompt or reward? (W2). Purely continuous reward, no tolerance band: coordinate arm still below its blind baseline (−0.026); marker arm ahead on coordinate-free prompts by +0.091 (p = 0.002, 3/3 seeds). The effect does not come from the band.

Probe (W3). Agreed; Sec. 4.1 now reads: the probe locates the bottleneck after perception; it does not show that the marker unlocks this same representation.

Why mkc < co on coordinate-free prompts? A model trained with coordinates in the prompt learns to rely on them, marker or not, and loses its main cue when they are absent (SFT mkc − co there: −0.041). This supports the decomposition: the active part is removing the coordinates (mk − mkc: +0.071 RLVR, +0.134 SFT, on identical images). Sec. 5 now says this.

InternVL (Q1). Its frozen features are the most depth-decodable of the three backbones (probe δ<1.25 0.3275 vs 0.3075 / 0.2225) at the same size as Qwen3-VL-8B, so neither a weaker vision tower nor capacity explains the failure; it lies in turning available features into a metric answer. We do not have evidence to separate decoder from training dynamics, and state the formulation effect as established for one model family and not reproduced on this second one.

No ground plane (Q2). The protocol does not assume one: the blind baseline is fitted to whatever text the benchmark gives, so where the text carries no depth it is simply low and any margin must come from the image. Indoors it flags our driving-trained arm as having no transferable metric ability (Sec. 7). The 180° rotation is a controlled version inside our data: it removes the usable ground-plane relation without changing the scene, and the marker models keep ranking by true depth.

Reviewer_boNCOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether replacing textual pixel coordinates with visual markers changes what spatial post-training learns. Using LiDAR-derived depth queries, training comparisons, a marker-plus-coordinate decomposition, and shortcut audits, it reports improved metric-depth performance with markers. It also documents important boundaries: persistent ordinal shortcuts, failed indoor metric transfer, and unsuccessful replication on another model family.

Critical strengths
  1. The empirical investigation is systematic and thorough. Paired-seed experiments, the marker-plus-coordinate control, adversarial evaluations, and tests across domains and model families provide complementary checks on the findings. The explicit reporting of failed indoor transfer, unsuccessful cross-vendor replication, and persistent ordinal failures further reflects a careful effort to test the limits of the proposed explanation. This breadth of validation makes the findings more informative and their scope more clearly defined.

  2. The paper presents a novel and insightful perspective on spatial reasoning in MLLMs. Its central idea about investigating how task formulation shapes what models learn is conceptually compelling. Treating query design as an variable provides a valuable perspective on shortcut learning, with meaningful implications for designing spatial reasoning benchmarks and training methods.

Critical weaknesses
  1. The evaluation format needs clarification. Table 1 evaluates all arms on a common benchmark whose blind baseline, 0.1197, corresponds to the “markers, no coords” version in Table 7. This appears to introduce a format shift for the coordinate-trained arm. Please clarify each arm’s test-time inputs and report results in both formats on matched queries to separate differences in learned ability from format transfer.

  2. The experimental setup is unclear. The main experiment is described as RLVR, but Table 1 uses the same marker results labeled as SFT in Tables 2 and 8. The paper also claims equal training data, yet Appendix J lists 3,000 marker samples and 8,000 coordinate samples. It is therefore unclear whether the two groups were trained under the same conditions.

Questions for authors

Please address the issues mentioned in the weakness section.

3 comments

Thank you. Both concerns were correct and are fixed.

Evaluation format (W1). Every checkpoint is now evaluated on every formulation of the same points, each against that formulation's own blind baseline. The matched comparison is summarized in Table 1; the complete crossed matrix (three training × three test formulations, RLVR and SFT) is Table 2. It separates the two effects you distinguish. Format transfer is real: every RLVR arm scores best on its own formulation, and under RLVR the coordinate arm's own-format score stays below what that format's text gives away (−0.055, −0.026). Measured where the text gives no location, marker training yields more image-based metric ability: +0.064 (RLVR), +0.091 (RLVR, continuous reward), +0.092 (SFT), 9/9 seed pairs.

Setup (W2). Identical 3,000 points, family mix, intrinsics, steps and recipe for all formulations; RLVR and SFT reported separately (general response, Sec. 4.2).

On the RLVR–SFT difference you may notice in Table 1: SFT lets the coordinate arm exceed its blind baseline (+0.137) while RLVR does not. RLVR can only reinforce answers the policy samples, and on coordinate prompts the text alone already produces answers inside the tolerance band often enough to be rewarded, so there is little pressure to look; SFT pushes every example to its exact label. The marker arm still learns more of the image-based ability under SFT (+0.092). Sec. 4.3 now states this.

The authors' response has addressed my concerns. I retain my score. Given the newly added experiments and results, I suggest marking these updates with \rev{} in the manuscript for easy identification.

Thank you for your positive feedback. We are glad that our additional experiments and clarifications have addressed your concerns.

Reviewer_PR7FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how the representation of queried image locations affects spatial intelligence learning in vision–language models. It argues that textual pixel coordinates expose a shortcut, whereas visual markers reduce access to this shortcut and encourage visual processing. It also compares coordinate-based and marker-based training, introduces a markers-plus-coordinates condition, and evaluates performance using blind baselines, worst-bucket accuracy, adversarial subsets, and cross-domain tests.

Critical strengths

Clear formulation of the problem. The ground-plane analysis provides a concrete geometric explanation for why textual pixel coordinates can reveal depth information without visual understanding. The blind baselines make this concern measurable.

Targeted construction of the worst-bucket criterion and off-ground set. The worst-bucket criterion helps distinguish broad coverage across depth ranges from performance concentrated around common target values. While selecting points for which ground-plane predictions substantially disagree with LiDAR ground truth (off-ground set) directly tests the hypothesized shortcut.

Critical weaknesses

SFT or RLVR. The main experiment is introduced in an RLVR setting, but the marker training results, including its constituent seed results, is explicitly identified as SFT in Table 2 and Appendix Table 8. It is therefore unclear which training paradigm produced each headline comparison. Additionally, Figures 1–2 appear to show zero accuracy in all depth buckets for the coordinate-trained model, whereas Table 1 reports non-zero aggregate accuracy. Please clarify whether the figures intended to indicate only a zero worst-bucket score, or whether they use different runs, samples, or metrics.

The experiments do not sufficiently distinguish geometric learning from training–evaluation formulation compatibility. The main comparisons evaluate models on a marker-based benchmark, which matches the marker-trained condition more closely. A complete crossed evaluation on identical images and target points is needed.

The off-ground result supports a narrower conclusion than "training removes the metric shortcut." On the off-ground subset, the marker model exceeds the reported blind and best-constant baselines. However, Table 3 does not report the paper’s paired coordinate-trained condition on the same subset.

The worst-bucket criterion is overinterpreted as evidence of capability. Worst-bucket accuracy is a useful diagnostic for uneven performance. However, the mathematical consequence of taking a minimum, not an unfairness in the metric is that, the worst-bucket accuracy is always lower than aggregate accuracy. Comparing with models with very small aggregated accuracy is inheritably unfair. Another concern is the interpretation of a non-zero value as evidence that the model has learned geometry. A single successful prediction can make a bucket non-zero; random or distribution-based predictions can also yield non-zero accuracy in every bucket. Conversely, zero observed successes do not imply zero underlying success probability. Increasing the sample size improves statistical resolution, but does not turn this criterion into a sufficient test of geometric understanding.

Questions for authors
  1. Can the authors provide a complete crossed formulation evaluation (Table 2)? Train on markers only, then test on markers + coords; Train on markers + coords, then test on markers + coords.

  2. (Whether dependence on visual evidence) Can image-ablation tests and paired 180° rotation tests, with horizontal flips as a comparison, distinguish sensitivity to relevant image evidence from reliance on screen position? Please assess correctness and directional errors, rather than prediction consistency alone.

  3. Can the author report per-bucket counts, accuracies, confidence intervals, and blind/random baselines, and reconcile the all-zero bucket figures with the non-zero aggregate results?

  4. Which headline results come from RLVR versus SFT? Please clarify why the results in Tables 2 and 8 are identified as SFT, and reconcile the claimed equal training sizes with the 3,000 versus 8,000 queries listed in Appendix J.

2 comments

Thank you for a review that found real errors. Point by point:

SFT or RLVR; 3,000 vs 8,000 (W1, Q4). Both correct; see the general response. The new Table 1 reports RLVR and SFT separately, each with three seeds on identical data, and pairs only within a paradigm.

Figures 1–2 vs Table 1 (Q3). Figure 1(b) was wrong: in the submitted arms only the nearest bucket (1.7–5.5 m) was zero in every seed (0/48); every other bucket had hits in at least one seed. Both figures are now drawn from the matched per-bucket data. App. Q gives, per bucket: item counts (118–121), accuracy with Wilson 95% intervals, and the blind, best-constant and random references scored on the same items.

Crossed evaluation (W2, Q1). Done for every checkpoint on every formulation of the same points (full matrix in Table 2), including your two cells: train mk → test mkc 0.214 (RLVR) / 0.340 (SFT); train mkc → test mkc 0.297 / 0.399. The no-coordinates column separates learned image-based ability (+0.064 RLVR, +0.091 RLVR continuous, +0.092 SFT; 9/9 seed pairs) from format compatibility (the diagonal).

Off-ground with the coordinate arm (W3). Done. On matched data both arms exceed the off-ground blind baseline, and tested without coordinates the marker arm is higher: RLVR +0.033 (t = 3.49, p = 0.073), SFT +0.029 (t = 4.11, p = 0.054), all seed pairs positive; continuous reward +0.010 (p = 0.43, n.s.). With three seeds we present these as consistent directional results. We adopted your narrower phrasing (Sec. 6.2): the marker formulation yields more metric ability that survives removing the ground-plane cue.

Worst bucket (W4). We agree with each point and now say so in Sec. 3: it is a necessary diagnostic, not a sufficient test; a constant scores in the bucket that contains it (the best constant scores 0.60 in our second bucket); zero hits do not mean zero probability. Claims now rest on the margin over a blind baseline fitted to the same prompts, with the worst bucket reported alongside, with intervals.

Image ablation, 180° rotation, horizontal flip (Q2). Done for every checkpoint, scoring accuracy and error direction (general response, part 2; App. Q Table 9).

Dear Reviewer PR7F,

As the discussion period draws to a close, we would like to kindly follow up on our response.

We have conducted additional controlled experiments and revised the manuscript to address the concerns raised during the review. We hope that the new evidence and clarifications have addressed your concerns, and we would greatly appreciate your assessment of the revised findings.

Thank you again for your time and valuable feedback, which has helped us improve the paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2): Matched-data replication and the two errors we fixed

Summary of the rebuttal. We thank all three reviewers. We ran every experiment requested: 21 new training runs (RLVR 9, RLVR continuous reward 6, SFT 6; the SFT marker arm reuses the submitted seeds) and a crossed evaluation of 25 checkpoints on 24 evaluation sets. A revised PDF is uploaded; all changes are listed at the end of part 2.

The headline, on matched data. Under RLVR, a model trained on coordinate prompts learns no more than a text-only regressor already gets from those prompts: all six coordinate runs (two reward types × three seeds) score below that regressor, while every marker run exceeds its own text-only baseline by 0.12–0.18. On prompts that name no pixel, marker training yields more metric ability than coordinate training under RLVR, RLVR with a purely continuous reward, and SFT — nine of nine seed pairs.

Two errors in the submission, which PR7F and boNC found. (1) The marker arm of the submitted Table 1 was the three SFT seeds of Table 8, while one coordinate seed was RLVR. (2) The two arms were trained on different point pools (3,000 vs 6,000/8,000 queries) with different family mixes, and only marker prompts gave the intrinsics. Both are fixed by the design below; the submitted table is kept in App. Q.

Matched design. The same 3,000 training and 1,200 validation points are rendered three ways — mk (marker, no coordinates), mkc (marker and pixel coordinates), co (original image, pixel coordinates) — with identical ground truth, family mix and intrinsics. Each is trained with the submitted recipe (RLVR 100 steps; SFT 300), three seeds, and every checkpoint is evaluated on every formulation. Each formulation has its own text-only blind baseline, refitted on its own prompts: giving coordinates raises it from 0.120 to 0.297 on identical points.

New Table 1 (metric pass@1, 600 points, 3-seed means; margin over the blind baseline of the test prompt in parentheses):

paradigmtrained ontest: no coords (blind .120)test: coords (blind .297)worst bucket, no coordsoff-ground, no coords (blind .082)
RLVRcoordinates0.190 (+.070)0.242 (−.055)0.0260.123
RLVRmarkers0.254 (+.135)0.2020.0830.156
RLVR, continuous rewardcoordinates0.191 (+.071)0.272 (−.026)0.0210.143
RLVR, continuous rewardmarkers0.282 (+.162)0.1910.1320.153
SFTcoordinates0.199 (+.079)0.435 (+.137)0.0220.151
SFTmarkers0.291 (+.172)0.2580.1580.180

Paired marker − coordinate on the no-coordinates test: RLVR +0.064 (t = 3.91, p = 0.060), RLVR continuous +0.091 (t = 22.57, p = 0.002), SFT +0.092 (t = 3.78, p = 0.063; three seeds, df = 2); worst bucket +0.057, +0.111, +0.136; every seed pair has the same sign.

Reading. The no-coordinates column measures the ability that does not rest on the text. The diagonal shows format transfer is real — every RLVR arm scores best in its own format (the SFT marker arm does better still with coordinates added: 0.340, Table 2) — and under RLVR the coordinate arm's own-format score stays within what that format gives away. The claim, stated precisely: the formulation decides how much of the metric ability RLVR learns comes from the image. This is the paper's conclusion, now measured on matched data across formulations.

AuthorsOpenReview ↗

General Response (2/2): Image controls, reward, decomposition, and list of changes

Does the marker just expose its row? (dwyY W1, PR7F Q2). Three image controls on every checkpoint (App. Q, Table 9). Spearman ρ of predictions with LiDAR depth and with the depth the ground-plane relation gives the displayed row:

upright acc / ρ_LiDAR / ρ_rowh-flip acc180° rotation acc / ρ_LiDAR / ρ_rowgrey acc / ρ_row
RLVR markers0.254 / .81 / .810.2590.132 / .39 / −.120.089 / .24
RLVR coordinates0.242 / .78 / .830.2510.125 / .29 / −.090.043 / .29
SFT markers0.291 / .85 / .730.2770.197 / .60 / −.180.043 / −.11
SFT coordinates0.435 / .88 / .760.4320.154 / .39 / .010.144 / .51

Upright, true depth and the row formula are nearly collinear — that is why the shortcut works. Rotation reverses the row cue: the marker models keep ranking by true depth and anti-correlate with the row formula. A grey image keeps only the marker's position: the marker models drop below the blind baseline, while the coordinate SFT model still answers by the stated row (ρ_row = .51). On off-ground points, the submitted marker RLVR model is within tolerance of LiDAR 6× more often than of the formula (0.183 vs 0.030; joint-regression weight 0.38 on LiDAR depth vs 0.19 on the formula — a residual use of the row, which we report).

Is it the tolerance-band reward? (dwyY W2). With a purely continuous reward exp(−2|log(1+e)|) the coordinate arm still stays below its blind baseline (−0.026) and the marker arm leads on coordinate-free prompts by +0.091, with the most consistent seeds (p = 0.002). SFT, with no band at all, orders the arms the same way.

Decomposition on identical images (new Table 2). mk − mkc (same images, only the prompt differs) on coordinate-free prompts: RLVR +0.071 (p = 0.039), SFT +0.134 (p = 0.029), all seeds positive; mkc − co (marker drawn or not, coordinates in both): −0.006 and −0.041. The active part is that the prompt stops naming the location.

Changes in the revised PDF. Table 1 and Table 2 replaced by the matched designs; Fig. 1 regenerated from matched per-bucket data (the submitted Fig. 1(b) drew the coordinate arm as zero in every bucket, which was wrong — only its nearest bucket was zero in every seed); Fig. 2(b) likewise; Sec. 4.2 states data, steps and seeds per arm; Sec. 3 states that the worst bucket is a necessary diagnostic, not a sufficient test; Sec. 6.2 narrowed as PR7F suggested; the probe sentence narrowed as dwyY suggested; the error is listed among our corrections (Sec. 8); new App. Q with per-seed results, image controls, a per-bucket table with Wilson intervals and per-bucket blind/constant/random references, and the submitted comparison for reference; appendix tables and figures that still show submitted-design arms (Tables 3, 4, 5, 8; Figs. 4, 5, 6, 7, 13) are now labelled as such. Main text is within 10 pages. Code, data and all 25 checkpoints will be released.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.