acceptodds
Withdrawn from ICLR 2027

Making Localization Thinkable: Grounding Actions as Thought Primitives

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning has become fundamental to modern Vision-Language Models (VLMs), extending visual understanding beyond direct prediction. In visual grounding, however, reasoning remains largely semantic, leaving precise localization outside the reasoning process. We term this gap Reasoning-to-Coordinate Collapse: VLMs can identify the correct target yet still predict inaccurate coordinates because localization is not explicitly modeled during reasoning. We present ThinkAxis, a paradigm that brings localization directly into the reasoning process. ThinkAxis represents localization updates as executable grounding primitives and carries the evolving localization throughout the reasoning trajectory, enabling VLMs to reason over and act on coordinates alongside semantic interpretation. To optimize this localization reasoning process, we further investigate the visual evidence behind localization decisions and find that accurate coordinate evolution increasingly relies on boundary-relevant evidence. Building on this observation, we formulate a trajectory optimization algorithm, ERPO, which couples geometric progress with counterfactual evidence attribution. Extensive experiments across visual grounding tasks show the effectiveness of ThinkAxis, with particularly notable improvements in precise localization under strict localization criteria. These results highlight the potential of localization-space reasoning for broader multimodal systems where precise spatial grounding is critical.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Multimodal Reasoning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6FFuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem of visual grounding. To this end, the authors propose to incorporate the localization into the thinking / reasoning process to help a better and more precise visual grounding. The authors identify Reasoning-to-Coordinate Collapse as an essential gap between semantic reasoning and coordinate formation, and this is the motivation for their method. The authors develop E2RPO (edge-evidence attribution) to jointly optimize spatial updates and action-relevant visual attribution. The authors report results on several benchmarks, including RefCOCO, segmentation, RealVLG for robotic manipulation, etc. The results on RefCOCO show their improvement over Qwen2.5-VL-7B and Qwen3-VL-8B.

Critical strengths
  1. Visual grounding is an important task to study. As an essential capability of VLMs, the visual grounding capability can have lots of downstream applications such as robotics, autonomous driving and tool-based visual reasoning.

  2. Results on different domains. In addition to the conventional RefCOCO benchmarks, the authors have also shown the results of their method on other different domains, including driving and robotics, which is good to reveal the potential applications and impacts of the research.

Critical weaknesses
  1. Motivation of incorporating localization into the reasoning process is not very clear to me. While in driving scenarios such as Figure 15, I think it is important to incorporate localization into the reasoning process, I am not sure whether it is very effective in RefCOCO scenarios such as in Figure 4. In that example, the query is "the bus that is almost out of view", and the initial grounding bounding box incorrectly locates another bus; this is due to the incorrectness in understanding which target object the query actually refers to. If the model can understand the reference correctly, for example, it should be the bus on the left of the image instead, then it can definitely do the visual grounding quite correctly. Using the current paradigm as in Figure 4 can make the eventual prediction correct, but the process may not be the most efficient, as it involves getting the incorrect prediction first, delete it, and then add another prediction box.

  2. Efficiency results. The important applications of visual grounding models is in the robotics and autonomous driving scenarios. In these scenarios, being real-time is very important. Therefore, it would be better if the authors can also show the efficiency results of their method, to make sure that the visual grounding algorithm they developed is efficient for real-time deployment on embodied systems.

Questions for authors
  1. As mentioned in Weakness 1, would the proposed method / thinking trace actually be the most efficient way to fix those incorrect visual grounding samples?

  2. What is the efficiency of the proposed method compared with Qwen3 / Qwen2.5 models?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6FFu (1/2)

Thank you for your constructive comments. We have thoughtfully addressed all of your suggestions and concerns in our revised manuscript.

W1 / Q1. Motivation and Efficiency of Localization Reasoning

We clarify that ThinkAxis does not require an incorrect initial prediction followed by correction. Figure 4 is intended to illustrate the range of executable grounding actions, rather than a mandatory error-then-correction process. Instead, ThinkAxis adaptively determines whether and how to refine the current localization. As shown in the table below, 94.1% of predictions with an initial IoU ≥ 0.95 terminate directly without additional corrections, while more challenging cases receive progressively more refinement steps.

Initial IoUDirect Stop (%) ↑Avg. Steps ↓
< 0.512.03.64
[0.5, 0.95)48.02.20
≥ 0.9594.11.06
Overall52.22.20

Additional visualizations further illustrate this adaptive behavior. Figure 8 of the revised manuscript shows precise boundary refinement after correct target identification (IoU: 0.646 → 0.992), while Fig. 12 demonstrates semantic reassessment and target replacement (IoU: 0.039 → 0.987). Table 7 further shows that correction magnitudes decrease from 11.44% to 1.54% as localization accuracy improves, consistent with progressively finer adjustments. We have added the adaptive stopping analysis and precise localization example to Sec. 4.3 (Table 9 and Fig. 8) of the revised manuscript.

Regarding effectiveness on RefCOCO, the advantage of ThinkAxis is particularly evident in more precise boundary localization. Conventional evaluations primarily focus on [email protected], which does not fully capture the precision of predicted bounding boxes. ThinkAxis achieves an average +6.5 point improvement in [email protected] (Table 1), demonstrating the value of explicit localization reasoning for precise coordinate formation beyond coarse target identification.

AuthorsOpenReview ↗

Response to Reviewer 6FFu (2/2)

W2 / Q2. Inference Efficiency

As suggested, we evaluate inference efficiency on RefCOCOg using Hugging Face Transformers with BF16 precision and FlashAttention-2 on NVIDIA A800 80GB GPUs, with a per-GPU batch size of 4 and a maximum generation length of 1,024 tokens. Under these settings, we report the average number of visual encodings and inference latency per sample for the original Qwen backbones, iterative baselines, and ThinkAxis.

MethodAvg. Visual Encodings ↓Avg. Latency (s) ↓
Qwen2.5-VL-7B1.01.90
Qwen3-VL-8B1.02.47
IVT3.28.90
Multi-turn2.46.07
ThinkAxis-7B (Ours)1.03.28
ThinkAxis-8B (Ours)1.03.92

Compared with the original Qwen2.5-VL-7B and Qwen3-VL-8B backbones, ThinkAxis incurs additional inference latency (1.90s → 3.28s and 2.47s → 3.92s, respectively) due to explicit localization reasoning, while achieving substantial gains in localization precision, including an avg. +6.5 point improvement in [email protected] over Qwen2.5-VL-7B. We have included the inference-efficiency comparison in Sec. 4.3 (Table 8) of the revised manuscript.

Importantly, ThinkAxis avoids repeated visual encoding across localization steps. Compared with the evaluated IVT-style and multi-turn baselines, ThinkAxis-7B reduces inference latency by 63.1% and 46.0%, respectively, while achieving higher localization accuracy. Furthermore, revised Appendix Table 13 (original Table 11) reports only 2.2 average localization steps, compared with 3.4 for raw offsets and 4.2 for absolute-coordinate regeneration, demonstrating the efficiency of structured grounding actions.

Regarding practical applications, current VLMs still face challenges in meeting the real-time requirements of autonomous driving and robotics. We see greater near-term potential in high-precision offline data annotation for both domains, particularly in complex driving scenes with densely distributed objects and robotic manipulation scenarios requiring precise object boundaries, where direct coordinate prediction remains challenging.


Summary. Our additional analyses clarify that ThinkAxis does not require an incorrect initial prediction or a fixed correction process, but adaptively determines when and how localization should be refined. The efficiency experiments quantify its computational overhead over the original Qwen backbones while demonstrating the benefits of avoiding repeated visual encoding compared with multi-turn approaches. Together with the driving experiments, these results further support the practical value of ThinkAxis for high-precision offline annotation.

Reviewer_jTHMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ThinkAxis, which interleaves textual reasoning with executable bounding-box editing actions, including moving boundaries, adding boxes, deleting boxes, and terminating refinement. The model performs successive corrections under a shared visual encoding. Training combines synthetic correction trajectories for supervised fine-tuning with E²RPO, a GRPO-based method incorporating geometric credit assignment and counterfactual visual-evidence weighting. Experiments evaluate referring expression comprehension, segmentation transfer, visual reasoning, and application-oriented grounding.

Critical strengths
  1. The method addresses a practically relevant objective: improving localization precision beyond identifying the correct target. Its executable action representation is concrete, and sharing a visual encoding offers a potentially useful efficiency advantage.
  2. The experiments demonstrate improvements over both evaluated backbones, particularly under strict localization thresholds. The representation ablations and evaluations across several tasks provide supporting evidence of practical utility.
Critical weaknesses
  1. The conceptual and methodological advances are insufficiently established. The observation that correct textual target descriptions can coexist with inaccurate boxes does not adequately establish a distinct “Reasoning-to-Coordinate Collapse” mechanism or identify its cause. Spatial perception, coordinate representation, and numerical output remain plausible explanations. Meanwhile, GETok already studies spatial tokens and relative-offset refinement, and IVT combines synthetic correction trajectories, supervised training, and GRPO with visual feedback. ThinkAxis performs genuine coordinate updates, but its particular advantage from omitting refreshed visual feedback needs a clearer controlled comparison. E²RPO also builds on ideas related to visual-dependence objectives and differentiated credit assignment in PAPO and SRPO, without directly establishing its additional benefit over these approaches.
  2. The empirical comparisons do not sufficiently establish the method’s advantage over strong existing alternatives. In Table 1, ThinkAxis-7B achieves 89.2 average [email protected] versus 88.9 for Ground-R1 and 88.8 for GETok. The larger strict-localization gains are measured against the original backbone, without corresponding strong-method comparisons. Table 2 omits GETok, whose published segmentation results are competitive and higher on several listed splits. More importantly, Table 3 includes the RealVLG-R1 GRPO variant but omits its stronger GSPO variant: the latter reports 88.5/88.9 gIoU/cIoU on Novel, compared with 85.1/85.8 for ThinkAxis. These omissions materially affect the interpretation of the reported advantages. The training and evaluation differences should be clarified.

[1] GETok: Grounding Everything in Tokens for Multimodal Large Language Models

[2] IVT: Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding

[3] RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation

Questions for authors
  1. Can you provide a controlled comparison with IVT using matched backbones, training data, and inference budgets? What evidence distinguishes the proposed mechanism from a useful reparameterization of coordinate prediction?
  2. Why are GETok’s segmentation results and the RealVLG-R1 GSPO variant omitted? Please include these comparisons or explain relevant protocol differences, and report strong grounding baselines under strict IoU thresholds.
5 comments
AuthorsOpenReview ↗

Response to Reviewer jTHM (1/3)

Thank you for these valuable comments and suggestions. We have clarified these points and incorporated the corresponding revisions into the manuscript.

W1. From semantic identification to executable localization reasoning

We address these concerns in three parts:

  • First, we clarify that Reasoning-to-Coordinate Collapse describes an observed failure mode rather than the sole cause of localization errors. This observation motivates us to investigate how reasoning can directly contribute to precise coordinate formation, beyond semantic understanding alone. We acknowledge this concern and have revised Sec. 1 to make this distinction explicit.

  • Second, as discussed in Sec. 2 and Fig. 2, ThinkAxis differs from GETok and IVT in how localization is integrated into the reasoning process. GETok explores coordinate representations through grid tokens and relative offsets, while IVT employs iterative correction through visual feedback. In these approaches, reasoning primarily focuses on semantic interpretation or visual observations, with coordinate updates generated as predictions rather than explicitly reasoned over within the reasoning trajectory. In contrast, ThinkAxis incorporates localization directly into reasoning, enabling the model to reason about and execute coordinate adjustments within a unified autoregressive trajectory. We further clarify this distinction and provide controlled comparisons with IVT in our response to Q1 below.

  • Third, we clarify the distinction between E²RPO and PAPO/SRPO in the third subsection of Related Work. E²RPO specifically targets boundary-relevant visual evidence for precise localization, coupling evidence dependence with the geometric progress of executable actions. In contrast, PAPO encourages general visual dependence, while SRPO assigns visual-dependence-aware credit across different token roles. Neither explicitly connects boundary-relevant evidence to the geometric progress of localization corrections.

AuthorsOpenReview ↗

Response to Reviewer jTHM (2/3)

Q1. Controlled Comparison with IVT and Localization Reasoning

(i). Controlled comparison with IVT.

IVT investigates visual-feedback-based localization correction using a 4B backbone, only 2.4K training examples, and 505 test samples, a relatively limited setting compared with standard visual grounding benchmarks. Its GRPO reward primarily evaluates final-box IoU and format validity, without explicitly rewarding intermediate corrections. Although no official implementation is publicly available to our knowledge, its rendered-box feedback strategy warrants further evaluation. We therefore adapt this strategy into a multi-turn baseline using the same backbone, SFT training data, and training budget as ThinkAxis.

MethodAvg. Visual Encodings ↓[email protected] ↑[email protected] ↑Avg. Latency (s) ↓
IVT3.284.952.18.90
Multi-turn2.485.253.26.07
ThinkAxis (Ours)1.086.456.33.28

Under matched training settings, ThinkAxis outperforms IVT-style baseline by +1.5 points in [email protected] and +4.2 points in [email protected], while reducing inference latency by 63.1%. Compared with our internal multi-turn baseline, ThinkAxis also improves [email protected] by +3.1 points with 46% lower latency. These results demonstrate that explicit localization evolution within a single autoregressive trajectory achieves both more accurate and more efficient refinement without repeated visual encoding.

(ii). Distinction from iterative coordinate refinement.

We clarify that ThinkAxis is not merely a different representation of coordinate prediction, but a formulation that brings localization directly into reasoning within a unified autoregressive reasoning trajectory. Unlike multi-turn approaches that rely on repeated visual feedback and re-encoding to drive successive corrections, ThinkAxis integrates localization directly into the reasoning process through three aspects:

  1. Reasoning: Rather than producing coordinates as the outcome of each reasoning round, the model explicitly reasons about how the current localization should evolve.
  2. Execution: Grounding actions directly modify the current spatial hypothesis, allowing localization and semantic reasoning to jointly evolve within a single autoregressive trajectory.
  3. Optimization: Each action produces a verifiable geometric transition, enabling E²RPO to optimize intermediate corrections through geometric progress and boundary-relevant visual evidence.

Our ablations separately assess coordinate representation and localization-conditioned reasoning. Sec. 3.1 (Eq. 1 and Fig. 3) analyzes why relative actions enable more effective corrections, while the revised Table 13 (original Table 11) confirms their benefits over absolute-coordinate regeneration ([email protected]: 52.7 → 56.3; Avg. Steps: 4.2 → 2.2). More importantly, the revised Table 11 (original Table 9) shows that localization-conditioned reasoning provides additional gains even under the same action representation, improving [email protected] by +1.2 points, [email protected] by +0.5 points, and ReasonSeg gIoU by +2.6 points over Action-only. These results support a contribution from localization-conditioned reasoning beyond coordinate reparameterization.

AuthorsOpenReview ↗

Response to Reviewer jTHM (3/3)

W2 / Q2. Stronger comparisons at standard and strict localization thresholds

We provide additional evaluations and clarify the differences in experimental protocols below.

(i) Strong baselines under strict IoU thresholds.

Our main-table comparisons prioritize evaluation fairness and consistency. GETok and Ground-R1 have not released their model weights, preventing direct evaluation under our strict-IoU protocol. GETok reports [email protected] under a different evaluation protocol, rather than the full RefCOCO/+/g splits used in our Table 1. We therefore avoid directly combining results obtained under different evaluation settings. To enable a more consistent comparison, we additionally evaluate ThinkAxis under GETok's reported [email protected] protocol, with results shown below.

MethodRefCOCO ValRefCOCO TestARefCOCO+ ValRefCOCO+ TestARefCOCOg ValRefCOCOg TestAvg.
VisionReasoner74.278.468.975.372.173.273.7
GETok-R175.680.171.678.274.876.176.1
ThinkAxis (Ours)76.880.974.380.474.976.577.3

ThinkAxis achieves the highest [email protected] across all six evaluated splits, outperforming GETok-R1 by +1.2 points on average. These results provide an additional comparison under the same reported evaluation protocol, particularly at a stricter localization threshold.

(ii) GETok segmentation results.

We clarify that Table 2 primarily evaluates whether precise box localization can transfer to dense segmentation without segmentation-specific training of ThinkAxis. ThinkAxis directly uses predicted boxes as prompts for SAM, whereas GETok employs dedicated segmentation supervision data and a different output representation. Given these differences in training objectives, we did not directly combine their segmentation results in the original table. We have clarified the purpose of this evaluation in Sec. 4.2 of the revised manuscript.

(iii) RealVLG-R1 GSPO comparison.

As suggested, we include the stronger GSPO variant of RealVLG-R1 to provide a more comprehensive comparison.

MethodSeen gIoU/cIoUSimilar gIoU/cIoUNovel gIoU/cIoU
RealVLG-R1 (GRPO)88.0 / 88.583.4 / 84.284.6 / 85.4
RealVLG-R1 (GSPO)89.0 / 89.686.4 / 86.988.5 / 88.9
ThinkAxis (Ours)89.3 / 89.688.0 / 88.585.1 / 85.8

Using less than one-third of the official training data, ThinkAxis achieves competitive performance on Seen and outperforms GSPO by +1.6 points in both gIoU and cIoU on Similar. GSPO performs better on Novel, where the gap may also be influenced by differences in training-data coverage and sampling. We have included GSPO in Table 3 and revised the corresponding discussion in Sec. 4.2.


Summary. The additional controlled comparisons and ablations clarify how ThinkAxis extends beyond coordinate representation by incorporating executable localization decisions into a unified reasoning trajectory. We further strengthen the empirical evaluation through an IVT-style comparison, strict-IoU results under GETok's protocol, and the inclusion of the stronger RealVLG-R1 (GSPO) baseline. Together, these results provide further evidence for the proposed localization reasoning formulation and a more comprehensive comparison with existing methods.

Reviewer_3WoZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ThinkAxis represents visual grounding as executable box-editing actions interleaved with language reasoning within one autoregressive trajectory. Geometrically spaced relative movements refine boundaries without repeated image encoding. Supervised trajectories are constructed from grounding annotations and supplemented with action-conditioned rationales. E2RPO combines response-level and route-level reinforcement learning with a bonus based on action-likelihood changes under visual masking. Experiments cover referring-expression localization, box-prompted segmentation, visual reasoning, and application-specific grounding. The results most clearly support action-based refinement; the additional contribution of textual reasoning and the evidence bonus needs firmer attribution.

Critical strengths
  1. A concrete and useful localization representation. The add/delete/move/end interface makes the geometric state explicit and executable while sharing one visual context. Table 11 supports the use of geometric movement tokens: [email protected] increases from 52.7 with absolute coordinates and 54.4 with raw offsets to 56.3, while average steps fall from 4.2 and 3.4 to 2.2. This provides evidence for a useful representation choice independent of how strongly the language trace is interpreted as reasoning.

  2. Gains are strongest where precise localization matters. Table 1 reports average [email protected] improvements from 86.6 to 89.2 for the 7B backbone and 88.7 to 90.8 for the 8B backbone. For the 7B setting, split-averaged [email protected] increases by 6.6, 6.8, and 6.2 percentage points on RefCOCO, RefCOCO+, and RefCOCOg. The box-prompted segmentation results and cross-backbone correction study provide additional evidence of practical value beyond a single localization metric.

Critical weaknesses
  1. The evidence does not yet isolate the claimed reasoning and optimization contributions. Table 10 does show that trajectory SFT and subsequent E2RPO each help, so a training-stage comparison is present. However, Table 1(b) reports RefCOCOg 88.6/58.2 at [email protected]/0.9, whereas the full setting in Tables 8–10 reports 86.4/56.3, without a sufficiently explicit mapping of backbone, split, and averaging protocol. Within Table 9, action-only already achieves 55.8 at [email protected] versus 56.3 for ThinkAxis; the larger ReasonSeg gain, 62.3 to 64.9 gIoU, is encouraging but does not resolve the small localization margin. Figure 7 compares optimization curves, but a matched final-test comparison with uncertainty is needed to determine the incremental benefit of the edge bonus. The unspecified SFT sample count and teacher/initializer identities further impede attribution to the proposed components.

  2. Mask sensitivity does not establish the claimed causal mechanism. Equation 8 evaluates the likelihood of a fixed original-image trajectory after removing visual patches. Appendix A.1 uses a one-cell boundary band and combines regions across routes within each response. Boundary and interior masks can differ in area and information content, particularly for small objects; a union mask also couples one route's score to evidence for other targets. In addition, the unchanged textual prefix can retain information obtained from the original image. The resulting score measures sensitivity under this intervention, but does not uniquely identify boundary evidence used to produce the correction. Table 6's per-route comparison is useful, yet larger causal-gap values are not independent validation when related likelihood sensitivity is rewarded during training.

Questions for authors
  1. What backbone, checkpoint, evaluation split, and averaging rule underlie Table 1(b) and Tables 8–11? Please reconcile the RefCOCOg values and identify the SFT sample count, frozen initializer, teacher, and training budget for these comparisons.

  2. Under that common protocol, what are the final held-out results for SFT, route-level RL without the evidence bonus, and full E2RPO? Please also quantify the uncertainty of the full-versus-action-only contrast. Figure 7 and Table 10 already provide useful partial evidence; a matched numeric comparison would determine how much benefit is attributable to the proposed additions.

  3. Does the evidence bonus improve held-out localization relative to an equal-area interior or random-mask control, and does that improvement persist with per-route masks? Please distinguish better localization from an increase in the optimized sensitivity score. A narrower interpretation of the causal claim would also address part of this concern.

  4. Appendix A.2 describes 2,805 images from 70 driving scenes as a development set. Which scenes are used for training, model selection, and Table 5 evaluation? This clarification is needed to assess the claimed application generalization.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3WoZ (1/2)

We thank the reviewer for the constructive feedback. We have carefully addressed each concern and revised the manuscript accordingly.

W1 / Q1. Evaluation protocol

We clarify that the full ThinkAxis results in Table 1(b) and Appendix Tables 8–11 use the same backbone, training data, supervision, training budget. The only difference between Table 1 and the appendix tables lies in the evaluation protocol.

  • For the main comparisons, Table 1 follows the standard expression-level evaluation on the official RefCOCO/+/g splits to ensure fair comparisons with existing methods.

  • For the appendix component ablations (original Tables 8–11; revised Tables 10–13), we use a fixed instance-level subset of RefCOCOg val, selecting one referring expression per target to reduce evaluation costs. Since each target can have multiple referring expressions, this avoids repeated autoregressive localization inference for the same target across multiple ablation configurations. Such instance-level evaluation is also used in recent visual grounding studies, such as VisionReasoner (ICLR 2026). The sample counts under the two evaluation protocols are summarized below:

Evaluation ProtocolImagesUnique ReferentsExpressions
Full expression-level (Table 1)1,3002,5734,896
Fixed instance-level (Tables 8–11)1,3002,5732,573

All ablation variants are evaluated on the same fixed subset, ensuring controlled comparisons. This distinction is now stated in Appendix A.2 of the revised manuscript.

  • Training details. The training setup and optimization hyperparameters are provided in Sec. 4.1 and Appendix A.1–A.2. Specifically, we construct 135K SFT trajectories from RefCOCO/+/g, using Qwen2.5-VL-7B as the frozen grounding initializer and Qwen3-VL-Plus as the reasoning teacher. Both SFT and RL are trained for one epoch, with the RL stage using 7K examples, a batch size of 16, and 8 rollouts per prompt.

W1 / Q2-a. E²RPO Ablation Results

As suggested, we report the final quantitative results for different optimization strategies using the evaluation protocol described in Q1, thereby complementing the optimization curves presented in Figure 7.

Training Strategy[email protected][email protected]
Trajectory SFT85.353.6
+ Route-level RL (w/o Edge)85.955.3
+ Full E²RPO86.456.3

Route-level optimization improves [email protected] by +1.7 points over trajectory SFT, and incorporating edge-evidence attribution yields a further +1.0 points. These results show that edge-evidence optimization provides additional gains beyond geometric route-level credit, particularly for precise localization. These results are now included in Appendix B.3 (Table 12) of the revised manuscript.

W1 / Q2-b. Localization Reasoning Analysis

We clarify that executable grounding actions and localization-conditioned reasoning in Table 9 (revised Table 11) are two complementary components of ThinkAxis, addressing precise coordinate refinement and semantic target understanding, respectively. The strong [email protected] performance of Action-only confirms the effectiveness of executable actions, which are themselves a central part of ThinkAxis. Localization-conditioned reasoning further helps the model identify errors in the current localization and select appropriate corrections based on the referred target. We further emphasize that the ReasonSeg results are particularly informative in this regard. Unlike conventional referring expressions, ReasonSeg often requires more complex semantic reasoning to identify the target. ThinkAxis achieves an additional +2.6 gIoU over Action-only on ReasonSeg, while Generic CoT provides little improvement. These results support the value of connecting semantic reasoning directly to localization decisions, beyond geometric refinement alone.

As suggested, we conduct a paired image-level bootstrap on RefCOCOg Val to assess the reliability of the strict-localization improvement. We resampled images with replacement while retaining paired predictions from both models, repeating this process 10,000 times. The resulting [email protected] improvement is +0.5 points, with a 95% confidence interval of [+0.12, +0.9] points, computed from the 2.5th and 97.5th percentiles.

AuthorsOpenReview ↗

Response to Reviewer 3WoZ (2/2)

W2 / Q3. Edge-Evidence Analysis

The evidence score in Eq. (8) measures the sensitivity of localization-action likelihoods to visual perturbations, conditioned on the same reasoning trajectory. By keeping the textual prefix fixed, we assess the dependence of a given action on visual evidence, rather than claiming to identify the unique causal mechanism underlying the decision. This interpretation is now explicit in Section 3.3.3 of the revised manuscript.

1. Boundary evidence and localization accuracy.

To verify that the evidence bonus improves localization rather than merely increasing the sensitivity score, we compare boundary masking with an area-matched interior control, using identical SFT initialization, RL training budgets, and evaluation protocols.

Evidence Strategy[email protected] ↑[email protected] ↑
Route-level RL (w/o Edge)85.955.3
Random Mask85.855.7
Interior Mask86.255.7
Boundary Mask (Ours)86.456.3

Boundary masking improves [email protected] by +1.0 point over route-level RL and +0.6 points over both random and interior masking. These results show that boundary-focused evidence optimization achieves higher final localization accuracy than the alternative masking strategies under the evaluated settings.

2. Per-route evidence evaluation.

We further evaluate the evidence bonus using separate masks for individual routes, instead of the sample-union approximation.

Attribution Strategy[email protected] ↑[email protected] ↑
Per-route86.156.0
Sample-union (Ours)86.456.3

Per-route attribution improves [email protected] by +0.7 points over route-level RL without evidence credit, confirming that the benefit of boundary-evidence optimization persists under independent per-route masking. Sample-union further improves [email protected] to 56.3 (+1.0 point over route-level RL). Notably, Table 6 shows that per-route attribution produces a higher evidence-sensitivity score but lower final localization accuracy than sample-union, indicating that stronger evidence sensitivity does not necessarily translate into better localization. These additional ablations are included in Appendix B.6 of the revised manuscript, with the masking comparisons reported in Table 15.

Together, these results support the effectiveness of boundary-relevant evidence for precise localization while distinguishing improvements in final localization accuracy from increases in the optimized sensitivity score.


Q4. Driving-scene partitions and the scope of Table 5

The driving experiments of Table 5 follow a strict scene-level separation between training, model selection, and evaluation. The training corpus contains 67,222 images from 1,646 driving scenes, with approximately 287.6K annotated bounding boxes. Model selection uses a separate validation split, distinct from the held-out evaluation scenes.

Importantly, the 70 driving scenes (2,805 images) used for Table 5 are entirely held out from both training and model selection. Therefore, Table 5 evaluates localization performance on previously unseen driving scenes, providing direct evidence of scene-level generalization rather than performance on previously observed environments.

We have clarified these dataset partitions and their respective purposes in Appendix A.2.


Summary. The additional analyses clarify the evaluation protocols and training details, disentangle the contributions of localization reasoning and E²RPO, and provide direct evidence of improved localization accuracy through controlled masking and per-route evaluation. We have also clarified the interpretation of visual evidence sensitivity and the scene-level separation in our driving experiments. We believe these additions provide a more rigorous and transparent evaluation of ThinkAxis.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank you for your careful evaluation and constructive feedback. Your insightful comments have helped us clarify the central perspective of ThinkAxis and strengthen its empirical evidence. In response, we have revised the manuscript and conducted additional controlled experiments to provide a more comprehensive evaluation of our method. Below, we highlight our research perspective and summarize the additional experimental evidence and key findings.

1. Research Perspective.

We clarify that our central contribution is not iterative coordinate refinement itself, but how localization is incorporated into and driven by reasoning. Our starting observation is that, in visual grounding, VLM reasoning primarily focuses on interpreting visual semantics, with little explicit reasoning about the coordinates themselves, such as why a boundary should lie at (x=10) instead of (x=12). Consequently, coordinates typically appear only at the end of reasoning, leaving their formation largely implicit and difficult to optimize through supervision on only the four final coordinates of a bounding box . Motivated by this observation, ThinkAxis explicitly brings localization into the reasoning process by formulating grounding actions as executable thought primitives, allowing semantic reasoning and localization to jointly evolve within a single autoregressive trajectory.

This formulation provides a distinct approach to iterative localization. As illustrated in Fig. 2 of our manuscript, existing multi-turn approaches typically rely on repeated visual feedback and re-encoding to guide successive corrections, with reasoning primarily focused on interpreting updated visual observations. In contrast, ThinkAxis maintains an evolving localization state under a shared visual context, enabling the model to reason directly about how the current coordinates should change and execute successive spatial actions, without repeated visual encoding. This is particularly valuable in complex scenes where direct coordinate prediction struggles to achieve precise localization. ThinkAxis enables adaptive reasoning for more accurate predictions without repeated visual encoding, avoiding the substantial computational overhead that accumulates across multiple visual interaction rounds.


2. Additional Experimental Evidence.

We provide further experiments and analyses to strengthen the evaluation of ThinkAxis in three aspects:

  • Mechanism validation: Additional ablations disentangling executable actions, localization-conditioned reasoning, and E²RPO, including boundary-evidence controls and per-route attribution.
  • Efficiency and generalization: Adaptive stopping analysis, inference latency comparisons with original backbones and iterative baselines, and clarification of scene-level generalization in autonomous driving.
  • Method effectiveness: Controlled comparisons with an IVT-style baseline, additional strict-IoU evaluations under GETok's protocol, and comparisons with the stronger RealVLG-R1 (GSPO) variant.

3. Key Findings.

These additional results strengthen three main conclusions:

  • Localization reasoning contributes beyond coordinate representation. Executable actions provide an effective basis for coordinate evolution, while localization-conditioned reasoning yields additional gains even under the same action representation. This distinguishes the benefits of explicit reasoning from those of coordinate reparameterization alone.
  • Boundary-relevant evidence contributes to precise localization. E²RPO improves final localization accuracy beyond geometric route-level optimization, with additional gains over random and interior masking controls. The improvements also persist under per-route attribution, demonstrating benefits beyond increased visual sensitivity alone.
  • Adaptive and efficient localization: ThinkAxis determines whether and how to update coordinates based on the current localization, rather than enforcing unnecessary corrections. It achieves pronounced improvements under strict IoU thresholds while avoiding repeated visual encoding, resulting in lower inference latency than the evaluated multi-turn baselines.

We have also clarified the evaluation protocols, training configurations, interpretation of evidence sensitivity, and scope of our experimental comparisons in the revised manuscript.

We sincerely appreciate your consideration and welcome further discussion.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on Rebuttal and Invitation for Further Discussion

Dear Area Chair and Reviewers,

We sincerely thank you again for your valuable feedback and constructive suggestions. We have carefully addressed the reviewers' concerns in our detailed responses and revised the manuscript with additional experiments and analyses.

As the discussion period is approaching its end, we would greatly appreciate it if the reviewers could revisit our responses and share any remaining concerns or questions. We would be happy to provide further clarifications and engage in additional discussion.

Thank you very much for your time and consideration.

Sincerely, The Authors

Area_Chair_thg6OpenReview ↗

Please engage in the discussion

Dear Reviewers,

Thank you for your reviews and contributions so far. The authors have now posted their response, and I would like to kindly remind you to review it and engage in the discussion where appropriate.

Please also consider whether the authors' clarifications address any of your concerns and update your assessment if warranted.

Thank you for your time and effort.

Best regards,

Your AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.