acceptodds
Under review as a conference paper at ICLR 2027

Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models

10 views

Abstract

Geospatial reasoning links VLMs to real-world action and decision making. Reliable reasoning requires accurate spatial information and effective spatial representations. However, current VLMs obtain this information mainly from language and vision. At larger geographic scales, limited visual observations make it difficult to establish accurate spatial references and recover precise spatial relations. This leads to a bottleneck for VLM spatial perception and reasoning. This paper proposes that geospatial vector geometry can become a native spatial modality within a VLM and introduces GeoVec-VLM. First, we develop a structured geometry representation that preserves the spatial structure and arrangement of vector objects, building an explicit spatial basis. Second, we introduce a geometry interaction strategy that brings this basis into multimodal computation, where it interacts with visual content and language references according to the queried spatial relations. Together, these designs allow perception and reasoning to operate on shared geometric references. Across five spatial reasoning tasks, GeoVec-VLM improves mean accuracy by about 11% relative to the strongest baseline, maintains its gains under geographic transfer, and shows greater geometric consistency. Our results show that geospatial vector geometry offers a promising internal spatial basis for VLM spatial reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-language models, geospatial vector data, spatial reasoning, vector geometry, multimodal representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Jnk6OpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GeoVec-VLM, which integrates geospatial vector geometry with raster images and language for spatial reasoning. Polygon boundaries are encoded into reference-point and path tokens, while object, role, and relation states organize their interaction with visual information and language queries. Auxiliary spatial objectives supervise these representations. Experiments evaluate several geometric reasoning tasks on geographically disjoint regions in China and transfer sets in the United States and Singapore, reporting improvements in accuracy and geometric consistency.

Critical strengths
  1. The empirical evaluation includes useful ablations and geographic transfer tests. Comparisons between pooled and element-level representations, together with ablations of spatial states and auxiliary supervision, help assess the proposed components. Evaluation on held-out regions and additional geographies also provides evidence beyond performance on the training distribution.
  2. The geometric consistency analysis is valuable. Evaluating reciprocal relations and agreement between categorical direction predictions and displacement estimates reveals limitations that individual-answer accuracy can overlook. This adds useful information about the models’ spatial behavior.
Critical weaknesses
  1. The empirical positioning against the closest geometry–language approaches remains incomplete. The paper discusses SLM, but Table 2 evaluates adaptations of geometry encoders rather than directly establishing the advantage over the full SLM approach. Although these comparisons are informative, they leave uncertainty about the incremental contribution relative to the closest prior work and the effect of differences in supervision and adaptation.
  2. The contribution of vision to the main reasoning tasks is insufficiently isolated. Several evaluated relations can be determined from the supplied vector geometry. The cross-modal grounding experiment supports the ability to associate visual and geometric information, but the additional benefit of imagery across the main tasks remains less clear. This limits how broadly the results support multimodal spatial reasoning.
Questions for authors
  1. How does the Geo2Vec-based variant in Table 2 relate to SLM’s integration and training procedure? Could you clarify which aspects are retained or changed and how this affects the comparison with the closest prior approach?
  2. Are auxiliary spatial supervision and adaptation budgets comparable across the interfaces in Table 2? Given their different geometry-pretraining procedures, how much of the observed advantage can be attributed specifically to retaining element-level structure?
  3. Could you provide geometry-only ablations for tasks whose necessary inputs are available in vector form, and quantify the additional benefit of imagery? This would help distinguish geometric reasoning improvements from improvements requiring visual–vector interaction.

I am not an absolute expert in this field. Answering the questions above would increase my confidence in accepting the paper.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Jnk6 (1/2)

We thank the reviewer for recognizing the value of our geometric-consistency analysis beyond answer accuracy, as well as our ablations and geographic transfer results.

W1 / Q1. How does the Geo2Vec baseline relate to SLM's integration and training procedure?

Technical Relationship and Comparison Scope. Our original Geo2Vec baseline was a component-level comparison using frozen object-level Geo2Vec embeddings, a learned projector, and rank-16 LoRA on Qwen3-VL-8B. It did not reproduce SLM's complete spatial adapter and spatial instruction training pipeline.

SLM integrates Geo2Vec's object-level geometry embeddings into a language model for geometric operations and spatial reasoning, while GeoVec-VLM researchs how vector geometry connects visual regions, geographic objects, and language queries to support multimodal spatial perception and reasoning. Since the original SLM does not incorporate visual observations and its official implementation was unavailable at submission, we initially compared its geometry encoder as a representative component. We have clarified these technical connections and the comparison scope in the revised Introduction (lines 51–54, 77–78) and Related Work (Section 2.2, lines 135–149).

Full SLM Comparison. Following the reviewer's suggestion, we adapted the subsequently released SLM implementation to our multimodal benchmark. We retained its three-layer geometry encoder, two-layer projector, spatial supervision, and alignment/joint training procedure, while replacing Qwen3-8B with Qwen3-VL-8B and adapting its spatial instructions to our task setting (Appendix E.2, lines 1802–1811).

Table R1. Comparison on the China benchmark (%; mean over five categories).

MethodDir.Prox.Topo.AreaDist.Mean
SLM (VLM-adapted)39.452.353.751.924.744.4
GeoVec-VLM80.480.781.177.865.977.2

Under the shared evaluation protocol, GeoVec-VLM performs better overall than VLM-adapted SLM. This comparison suggests the importance of effective alignment and interaction between visual and geometric information in multimodal spatial reasoning, where visual inputs can also influence spatial predictions. The contribution of element-level geometry is examined separately in W2/Q2.

W1/Q2. How much of the performance gain in Table 2 can be attributed specifically to element-level geometry under comparable supervision and training budgets?.

To enable a more controlled comparison, we equipped Geo2Vec and Poly2Vec with the O/A/Z spatial states, structured decoder, and auxiliary supervision following the design of Pool GeoVec-VLM, yielding Geo2Vec + spatial pathway and Poly2Vec + spatial pathway, and extended baseline training to up to 1,500 updates with validation-based checkpoint selection.

Table R2. Controlled comparison of geometry representations on the China test set (%).

Geometry interfaceDir.Prox.Topo.AreaDist.Mean
Geo2Vec (extended)31.052.864.655.523.445.5
Poly2Vec (extended)40.446.161.853.928.146.1
Serialized GeoJSON (extended)36.461.966.761.436.452.6
Geo2Vec + spatial pathway63.766.772.359.328.258.0
Poly2Vec + spatial pathway71.860.769.758.734.159.0
Pool GeoVec-VLM70.475.674.860.356.867.6
GeoVec-VLM80.480.781.177.865.977.2

The spatial pathway improves Geo2Vec and Poly2Vec by 12.5 and 12.9 pp, showing that part of the original performance gap was attributable to differences in spatial modeling and supervision. Nevertheless, both remain below GeoVec-VLM (77.2%). The original Pool–Full comparison provides more direct evidence for the benefit of retaining element-level geometry: preserving these tokens throughout multimodal interaction improves performance by 9.6 pp.

These additional comparisons have been incorporated into revised Table 2 (lines 408–420), with implementation details in Appendix B.4 (lines 1337–1344).

AuthorsOpenReview ↗

Response to Reviewer Jnk6 (2/2)

W2/Q3. Contribution of Visual Information to Spatial Reasoning

As reviewer's suggestion, we further anaylzed whether and to what extent the trained GeoVec-VLM relies on visual information for spatial reasoning. we replaced the raster image with a blank image at inference time, keeping the trained model, vector geometry, and language instructions unchanged. This intervention evaluates the model's dependence on visual input without retraining (revised Appendix E, Table 14, lines 1584–1603).

Table R3. Accuracy under visual removal on the China test set (%).

TaskOriginal imageBlank imageΔ (pp)
Direction80.469.5−10.9
Proximity80.771.7−9.0
Parcel–parcel topology79.5671.17−8.39
Point–parcel topology82.6747.28−35.39
Area77.858.1−19.7
Boundary distance65.938.5−27.4

Replacing the image reduces accuracy across all six tasks. Direction, Proximity, Parcel Topology, and Area decline by 8.4–19.7 pp, despite the relevant geometric information remaining available through vectors. Larger drops occur in Point Topology (35.4 pp) and Boundary Distance (27.4 pp), where blanking the image also removes the query-point crosshair.

Visual and Geometric Attribution. We are also happy to share with the reviewer our additional attribution analysis (revised Appendix E.1, Figs. 14–16). GeoVec-VLM shows visual responses over relevant parcel regions and boundaries, accompanied by element-level geometric attribution, particularly near the minimum-distance boundaries in Proximity. Together with the cross-modal grounding results in Fig. 4, these observations provide further qualitative evidence that visual information contributes to spatial grounding and reasoning.

Finally, we sincerely thank the reviewer for helping us clarify our research positioning and strengthen our experimental evaluation. We hope the additional experiments and explanations address your concerns and give you greater confidence in our work. We also wish you a good day.

Thank you to the authors for their thorough response and additional experiments. Most of my concerns have been addressed. I encourage the authors to incorporate these results into the final version of the paper. I maintain my positive score and have increased my confidence in my assessment.

Reviewer_xy7rOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeoVec-VLM feeds parcel boundaries into a VLM as element-level geometry tokens plus supervised spatial states, instead of serializing or pooling vector data. On its deterministic six-task benchmark it beats the strongest baseline configuration by about 11 percent relative at roughly 1/5 of the tokens, with the best geometric consistency (0.88). Ablations and interventions tie the gains to preserved element-level structure, within clean, aligned, 2D polygon scenes only.

Critical strengths
  • The paper's primary contribution is turning geospatial vector geometry into a native spatial modality inside a VLM. Boundaries are kept as element-level reference-point and path tokens rather than serialized or pooled, and learnable object, role and relation states make this structure readable and supervisable by the language path.

  • This paper proposes a large deterministic geospatial reasoning benchmark with six task families, 66,159 aligned image-vector scenes, 359,340 training QA instances, geographically disjoint splits and zero-shot U.S. and Singapore transfer, plus consistency diagnostics that show task accuracy can hide geometric inconsistency.

Critical weaknesses
  • How do the geometry vectors deal with the scale mismatch problem? Maps at different scales often make each parcel correspond to a different physical scale, so can the model handle inputs whose scales differ greatly?

  • In the comparison experiments, is the content in the GeoJSON equivalent to the information used to construct the geometry vectors, such as metric scale, angles and variable lengths? For a fair comparison, providing the same information used to construct the geometry vectors to the VLM in plain text JSON form would demonstrate the effectiveness of the method more directly.

Questions for authors
  • What is the scale range of the train or test images? Does the proposed method perform similarly on different scales of input maps?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xy7r

We thank the reviewer for recognizing our vector geometry representation and structured spatial modeling. We address the questions regarding map-scale robustness and the geometric information provided through textual interfaces with additional analyses.

W1/Q1. What is the scale range of the train or test images? How do the geometry vectors deal with the scale mismatch problem?

Robustness to Map-Scale Variation. GeoVec-VLM handles scale variation by preserving both normalized local geometry and metric spatial information (Appendix A.1, lines 711–777). Reference-point features encode object-relative positions and boundary progression, while metric offsets, path displacements, lengths, and bearings retain physical geometric information. The scene encoder further models cross-object relations in a shared metric coordinate system, allowing geometric relationships to remain accessible across different map extents.

Scale range. All raster scenes are rendered at 512 × 512 pixels, while their geographic extents vary. The 47,980 training scenes cover window side lengths of 0.368–28.396 km (median: 4.044 km), while scenes underlying the 10,800 China test questions span 0.647–50.859 km, including extents beyond the training range.

Scale-stratified evaluation. To examine performance across scales, we grouped test questions using the training-window tertile thresholds (2.93 and 5.78 km).

Table R1. GeoVec-VLM accuracy across map extents on the China test set (%).

Window side lengthMean accuracy
Small (≤ 2.93 km)81.0
Medium (2.93–5.78 km)79.6
Large (> 5.78 km)75.6

GeoVec-VLM maintains relatively stable performance across different map scales, with a maximum accuracy variation of only 5.4 pp. Notably, it still achieves 75.6% accuracy on large-extent scenes, demonstrating its ability to handle spatial reasoning across substantially varying geographic extents.

W2. Is the content in the GeoJSON equivalent to the information used to construct the geometry vectors, such as metric scale, angles and variable lengths?

Geometric Information Equivalence. Both methods derive from the same source parcel geometries. Serialized GeoJSON retains object identifiers, normalized vertex coordinates, and ordered polygon/ring structures, and the physical window side length (window_side_m), allowing metric distances to be recovered. GeoVec-VLM organizes geometry into structured reference-point and path elements, explicitly encoding geometry vectors. Thus, the interfaces share the same geometric source but differ in how spatial structure is represented and made accessible during reasoning.

Following the reviewer's suggestion, we further validate whether explicitly providing these attributes through geojson improves spatial reasoning. We conducted a zero-shot comparison using the original Qwen3-VL-8B-Instruct (to exclude the effects of task-specific fine-tuning), evaluating three textual geometry interfaces (see examples at https://anonymous.4open.science/r/supplement-2454/Material2.pdf):

  • Serialized GeoJSON: Original coordinate serialization.
  • Compact Nested: The same coordinates reorganized into compact nested arrays.
  • Explicit Elements: Compact Nested augmented with element types, ordering, displacements, lengths, bearings, and turning angles.

Table R2. Zero-shot accuracy (%) on the China test set and relative median geometry-text length.

Geometry inputDir.Prox.Topo.AreaDist.MeanLength†
Serialized GeoJSON32.1756.5650.7560.2233.5646.71.00×
Compact Nested32.7256.3950.2863.3336.4447.80.82×
Explicit Elements29.2857.6151.4558.0032.7245.82.50×

† Relative median geometry-text length, measured on the training set and normalized to Serialized GeoJSON.

Compact Nested reduces median geometry-text length to 0.82× the original while modestly improving mean accuracy from 46.7% to 47.8%. In contrast, Explicit Elements increases median length to 2.5×, incurring an additional 50.3M tokens over the test set, yet achieves only 45.8% mean accuracy. The substantial textual overhead of explicit attributes does not translate into consistent reasoning gains. These results suggest that effective spatial reasoning depends not only on the availability of geometric information but also on how it is structured and utilized within the VLM, further motivating our structure-preserving geometry representation.

Finally, your time and careful feedback are truly appreciated, which have helped us better evaluate GeoVec-VLM across map scales and clarify its comparison with GeoJSON. We have updated these experiments in our revised submission. We hope these additional experiments and explanations have addressed your concerns and given you greater confidence in accepting our work.

Thank you for the detailed responses. My concerns are addressed in the rebuttal and I will keep my rating.

AuthorsOpenReview ↗

Response to Reviewer xy7r

Thank you for your thoughtful follow-up and encouraging feedback. We appreciate your constructive suggestions, which have helped us further improve the manuscript.

Reviewer_VweGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GeoVec-VLM, which incorporates geospatial vector geometry into a vision-language model through structured point and path tokens. A hierarchical geometry encoder preserves object boundaries and scene-level spatial arrangement, while object, role, and relation states connect geometry with visual features and language queries. Training combines geometry pretraining, auxiliary spatial supervision, and language generation. Evaluation covers six task families across five spatial reasoning categories, using geographically disjoint Chinese regions and zero-shot transfer to the U.S. and Singapore. The model reports approximately 77–78% mean accuracy across these settings, improved geometric consistency, and gains on area and boundary-distance tasks withheld from downstream QA training.

Critical strengths
  1. The element-level vector representation is well designed, and clean internal evidence supports it. The method splits each polygon boundary into alternating reference-point and path elements. This representation keeps boundary order, ring membership, and local metric structure. The model does not pool these element-level tokens, so they stay available inside the VLM. Two results support this design. First, the pooled variant and the full model use the same encoder, spatial states, and training setup, and only the pooling step is different. The full model is 5.1–17.5 pp better on all five tasks (Table 2). Second, the inference-time token interventions agree with the task semantics (Appendix C.2). When the authors remove the scene reference, Direction and Proximity accuracy decrease the most (about −64 and −30 pp). When they remove the local shape, Parcel Topology and Area accuracy decrease, but Direction accuracy almost does not change. These trends are consistent across the three regions. This is the most reliable part of the paper. It directly supports the main claim that fixed-size object embeddings lose structure that spatial reasoning needs.

  2. The evaluation design is rigorous, and the consistency diagnostics have independent value. The authors split the data into geographically disjoint regions, and they pretrain the geometry encoder only on the training regions. The benchmark includes hard cases in which common proxies, such as centroid distance or bounding-box area, disagree with the exact relation. These cases reduce shortcut solutions. In zero-shot transfer to the U.S. and Singapore, mean accuracy stays at 77–78%. The paper also tests geometric consistency with three checks: paired reversible queries, agreement between direction and displacement, and swaps of candidate order. The results show that accuracy can hide inconsistent reasoning in frontier VLMs. For example, the direction answers of GPT-5.1 agree with its own displacement estimates in only 53% of cases. This diagnostic protocol does not depend on the proposed method. Other spatial reasoning benchmarks can use it, so it is a useful contribution to the community.

Critical weaknesses
  1. Table 2 is not a controlled comparison. The authors use Table 2 to isolate the benefit of element-level geometric structure. I do not think that it does this. If I read Appendix B.4 correctly, the Serialized GeoJSON, Geo2Vec, and Poly2Vec variants train only the projector and the LoRA modules. B.4 does not mention O/A/Z states, structured decoders, or L_select supervision for these variants. But these components are the core of GeoVec-VLM. Table 3 shows how much they contribute. When the authors remove O/A/Z from GeoVec-VLM, Proximity decreases to 0.507 and Topology decreases to 0.573. These values are almost the same as those of Serialized GeoJSON (0.517 and 0.569). Area decreases to 0.372, which is lower than GeoJSON (0.613). Thus, I think that most of the gain comes from the spatial-state modules and the auxiliary supervision, not from the representation itself. I ask the authors to give the baseline representations the same spatial states and supervision, or to make the claim weaker.

  2. The fine-tuned baselines perform near chance, which suggests insufficient training. After fine-tuning on the same training split, Serialized GeoJSON gets only 0.249 on eight-way Direction. This is lower than zero-shot Qwen3-VL-8B with GeoJSON input (0.322, Table 1). Proximity is a binary choice, and all three baselines get 0.50–0.52, which is close to chance. I find this result surprising. An 8B model should learn to compare centroid positions from normalized coordinates after this much supervision. I ask the authors to explain this result. I also ask them to confirm that they trained the baselines to convergence with a suitable configuration.

  3. A same-backbone, image-only baseline with the same training data is missing. Table 1 compares GeoVec-VLM with zero-shot API models and with spatial VLMs that use only 3,840 adaptation samples. No baseline fine-tunes Qwen3-VL-8B on the same data with image input only. Without this baseline, I cannot separate two sources of gain: the vector modality and supervised training on the task distribution. I ask the authors to add this baseline.

Questions for authors
  1. Do the Table 2 baselines use O/A/Z states, structured decoders, and L_select? If not, can you report results where these representations use the same spatial-state pathway and supervision as GeoVec-VLM?

  2. Fine-tuned Serialized GeoJSON (0.249) is worse than zero-shot GeoJSON (0.322) on Direction. Can you give the baseline training details (steps, learning rate, convergence, coordinate precision, truncation) and explain this result?

  3. Can you add a Qwen3-VL-8B baseline that you fine-tune on the same data with image input only? This result would separate the gain from the vector modality from the gain from task supervision.

5 comments
AuthorsOpenReview ↗

Response to Reviewer VweG (1/3)

We thank the reviewer for recognizing our element-level geometry representation, targeted interventions, and especially for noting the potential value of our geometric-consistency diagnostics to the broader spatial reasoning community. We address the reviewer's concerns with matched spatial-pathway comparisons, extended training, and an image-only same-backbone baseline.

W1 / Q1. How much of the performance gain in Table 2 comes from element-level geometry versus spatial states and auxiliary supervision?

New Controlled Comparisons. In the original Table 2, Geo2Vec and Poly2Vec lacked the O/A/Z spatial states and auxiliary supervision used by Pool and Full GeoVec-VLM. To address this difference, we equipped both baselines with the spatial pathway of Pool GeoVec-VLM, including structured decoders and auxiliary supervision, and extended baseline training (W2/Q2). These controlled comparisons have been incorporated into revised Table 2 (lines 408–420), with implementation details in Appendix B.4 (lines 1337–1344).

Table R1. Spatial reasoning accuracy on the China test set (%; mean over five categories).

Geometry interfaceDir.Prox.Topo.AreaDist.Mean
Geo2Vec (extended)31.052.864.655.523.445.5
Poly2Vec (extended)40.446.161.853.928.146.1
Serialized GeoJSON (extended)36.461.966.761.436.452.6
Geo2Vec + spatial pathway63.766.772.359.328.258.0
Poly2Vec + spatial pathway71.860.769.758.734.159.0
Pool GeoVec-VLM70.475.674.860.356.867.6
GeoVec-VLM80.480.781.177.865.977.2

The reviewer's observation about Table 3 is important. However, Fig. 5 shows that even without O/A/Z, structured readouts directly from geometry tokens G retain 70.3–80.2% accuracy, while language-answer accuracy drops substantially. This suggests that geometric information remains accessible in geometry tokens (G), but requires an effective spatial pathway to support language generation.

Consistently, adding the spatial pathway improves Geo2Vec and Poly2Vec by 12.5 and 12.9 pp, respectively, confirming that spatial states and supervision contribute to the original performance gap. More importantly, the controlled Pool–Full comparison isolates the contribution of element-level structure: preserving geometry tokens throughout multimodal interaction improves mean accuracy by 9.6 pp (67.6% to 77.2%). We have updated these extended training baseline in our revised submission.

AuthorsOpenReview ↗

Response to Reviewer VweG (2/3)

W2 / Q2. Were the baselines adequately trained, and why did fine-tuned GeoJSON initially underperform its zero-shot counterpart?

Training Configuration and Convergence. The original Table 2 caption and Appendix B.4 were intended to convey that the baselines followed an aligned training protocol. The baselines used Qwen3-VL-8B with frozen geometry encoders, learned projectors, and rank-16 LoRA. The LoRA learning rate (2×10⁻⁵) followed GeoVec-VLM Stage 1, while the global batch size (1,024) and updates (680) matched Stage 2. AdamW and the warm-up/cosine schedule were also aligned (Appendix A.3, Table 6).

Following the reviewer's suggestion, we extended baseline training to up to 1,500 updates, selecting checkpoints based on validation performance (see convergence curves at https://anonymous.4open.science/r/supplement-2454/converge_curve.pdf). The revised training protocol is documented in Appendix B.4 (lines 1342–1344).

Table R2. Accuracy changes after extended training (Δ pp; extended minus original).

MethodDir.Prox.Topo.AreaDist.Mean
Geo2Vec+10.7+1.7+12.0−1.2−1.2+4.4
Poly2Vec+17.1−4.2+9.3−2.7+3.3+4.6
Serialized GeoJSON+11.5+10.2+9.8+0.1+3.2+7.0

Extended training improves mean accuracy by 4.4–7.0 pp. Notably, Serialized GeoJSON's Direction accuracy rises from 24.9% to 36.4%, surpassing its 32.2% zero-shot performance. These results confirm the sensitivity of baseline performance to training duration, while the extended baselines remain below GeoVec-VLM.

Input Budget. Serialized GeoJSON uses scene-normalized vertex coordinates with six decimal places (15 for hard cases) and eight-decimal query-point coordinates. The maximum total input lengths are 4,664 tokens for training and 72,817 for evaluation, both within the 262,144-token context limit; no inputs were truncated.

Geometric Task Complexity. The reviewer's observation about centroid comparisons relates closely to our hard-case design (Appendix B.2, lines 1162–1179). We further clarify the underlying geometric challenges. For eight-way Direction, labels require exact area-centroid displacements; bounding-box centers or unweighted vertex averages can yield incorrect directions for irregular polygons. For Proximity, the target is minimum polygon-boundary distance, and our hard cases include candidate rankings that contradict centroid-distance rankings.

To evaluate reasoning beyond these approximations, we deliberately enriched the Direction and Proximity test sets with hard cases (80.0%/81.8%, versus 9.3%/14.5% in training). This tests generalization to configurations where simple geometric heuristics fail. All methods follow the same evaluation protocol. Detailed difficulty distributions, representative hard cases and visual/vector attribution are provided in the https://anonymous.4open.science/r/supplement-2454/Spatial_Reasoning_Benchmark_Preview/visual-attribution.pdf. For more hard cases visualization please refer to Appendix E.1 of our revised submission.

AuthorsOpenReview ↗

Response to Reviewer VweG (3/3)

W3 / Q3. How does GeoVec-VLM compare with a same-backbone image-only baseline fine-tuned on the same data?

Following the reviewer's suggestion, we trained an image-only Qwen3-VL-8B SFT baseline using the same downstream training split, supervised task families, and validation-based checkpoint selection. It achieves 47.9% mean accuracy, compared with 77.2% for GeoVec-VLM. The result has been added to revised Table 1 (line 358), with implementation details in Appendix B.4 (lines 1346–1349).

Table R3. Comparison with the same-backbone image-only SFT baseline on the China test set (%).

MethodDir.Prox.Topo.AreaDist.Mean
Image-only SFT30.650.766.462.929.147.9
GeoVec-VLM80.480.781.177.865.977.2
Δ (pp)+49.8+30.0+14.7+14.9+36.8+29.3

Visual and Geometric Attribution Analysis. We further analyzed answer-conditioned spatial attribution (Appendix E.1, Figs. 14–16). Interestingly, even when Serialized GeoJSON predicts correctly, its visual attribution remains concentrated around parcel labels, resembling its incorrect cases and the Image-only SFT baseline. In contrast, GeoVec-VLM attends more strongly to relevant parcel regions and boundaries, with corresponding element-level vector attribution, particularly in Proximity. We have updated these results in Table 1 of our revised submission.

This observation extends our geometric consistency analysis from answer-level reliability to the spatial evidence associated with predictions, offering qualitative insight into how structured vector tokens may guide visual spatial reasoning. We are pleased to share this finding with the reviewer and sincerely appreciate their suggestion, which motivated this analysis!

Finally, we sincerely thank you for the constructive feedback. We hope the additional experiments and clarifications address your concerns and strengthen your confidence in the contributions of our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviewers' Feedback and Authors' Response

We thank all reviewers for their constructive feedback and recognition of GeoVec-VLM's contributions. Their comments have helped us strengthen our experiments and improve the manuscript.

Strengths: Reviewers recognized GeoVec-VLM’s effective integration of vector geometry as a native spatial modality, combining a well-designed structure-preserving element-level representation with learned spatial states that connect geometry, vision, and language for spatial reasoning. They highlighted the strong empirical performance on geographically disjoint test regions and in zero-shot transfer to the U.S. and Singapore, as well as the rigorous ablation and targeted geometric interventions supporting the key design choices. All three reviewers emphasized the value of the geometric consistency analysis, which reveals contradictions across related spatial predictions that answer accuracy alone can miss.

We addressed the reviewers' main concerns as follows.

1. Gains from Structure-Preserving Vector Geometry (VweG, Jnk6).

  • Controlled comparisons and extended training: We added two new baseline variants by equipping Geo2Vec and Poly2Vec with the same spatial pathway used by Pool GeoVec-VLM. We also extended baseline training, examined convergence, clarified coordinate precision, and confirmed that no inputs were truncated. The results confirm the benefits of structure-preserving geometry representations and retaining element-level geometry tokens.
  • Image-only SFT: We added a same-backbone image-only baseline trained on the same data. Its considerably lower performance confirms the benefit of incorporating structured vector geometry.

2. Mulit-scale handling and information equivalence (xy7r).

  • Scale robustness: We conducted scale-stratified evaluations, showing that GeoVec-VLM maintains strong performance across substantially different geographic extents, including scales beyond the training range.
  • Textual geometry controls: We tested GeoJSON variants with explicitly provided geometric attributes. Adding lengths, bearings, and other geometric information yields no consistent improvement despite increased token consumption, highlighting the importance of how geometry is represented and utilized.

3. Comparison with prior work and Contribution of visual information (Jnk6).

  • Full SLM comparison: We adapted the subsequently released SLM implementation to our multimodal benchmark, retaining its core geometry encoding and training procedure. GeoVec-VLM substantially outperforms VLM-adapted SLM, strengthening the comparison with the closest geometry–language approach.
  • Image ablation: We replaced raster images with blank images while preserving vector geometry and language instructions. The resulting performance drops across all six task families demonstrate the contribution of visual information to spatial reasoning.

4. Additional insight from spatial attribution (VweG, xy7r, Jnk6).

  • Visual and geometric attribution: We further analyzed answer-conditioned spatial attribution and found that even when Serialized GeoJSON predicts correctly, its visual attribution remains concentrated around parcel labels, resembling its incorrect cases and Image-only SFT. In contrast, GeoVec-VLM shows stronger attribution to relevant parcel regions and boundaries, accompanied by corresponding element-level vector attribution, particularly in Proximity.
  • New insight: Complementing our geometric consistency analysis, this observation further reveals that correct answers may not reflect the spatial evidence relevant to the underlying geometric relations. It offers qualitative insight into how structured vector tokens may guide visual spatial reasoning toward geometrically relevant regions.

Overall, our additional experiments further support the contributions of structured vector geometry and vector–vision–language interaction, with GeoVec-VLM maintaining a clear advantage over all newly added baselines. We are grateful that the rebuttal discussions also inspired new perspectives on spatial reasoning.

Following our responses, VweG raised their rating from 2 to 3, xy7r confirmed that their concerns were addressed, and Jnk6 maintained a positive rating while increasing their confidence.

These revisions have been incorporated into the revised manuscript, including Tables 1–2, the Introduction and Related Work (Section 2.2), Appendix B.4, and Appendix E, with all changes highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.