acceptodds
Withdrawn from ICLR 2027

Fine-Grained Geometric Visual Perception: Recovering Precise Positions, Boundaries, and Trajectories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision–language models (VLMs) can recognize geometric objects and coarse relations, yet they remain unreliable when asked to recover precise positions, endpoints, boundaries, and continuous trajectories in a common canvas co- ordinate system. We formulate this capability as fine-grained geometric vi- sual perception (FGGVP) and introduce FGGVP-22K, a dataset containing approximately 20,000 synthetic samples and 1,500 real samples. The data cover explicit primitive localization, semantically specified components, and derived targets determined by geometric relations. We fine-tune Qwen3- VL-4B with native coordinate tokens and multitask supervision, combining autoregressive answer loss with ordinal-coordinate, expected pixel-distance, and coordinate-point CVaR losses. Experiments on synthetic, real, and ex- ternal geometric datasets show substantial improvements in fine-grained localization over the base model and stronger Image2Code reconstruction. The results indicate that targeted supervision of positions, boundaries, and trajectories improves geometric visual perception and transfers to down- stream reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Precise Geometric Vision, Semantic Geometric Grounding, Continuous Coordinates, Vector Reconstruction, Constraint-Aware Editing, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yM5COpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces fine-grained geometric visual perception (FGGVP), a capability that the authors argue is insufficiently captured by existing general-purpose VLMs. To address this limitation, the authors construct the FGGVP-22K dataset and fine-tune Qwen3-VL on it. Experimental results show that the resulting model achieves stronger performance on FGGVP-related tasks, outperforming several frontier foundation models, including GPT-5.6 and GLM-5.3.

Critical strengths

This paper raises a potentially meaningful topic, namely the distinction between recognizing geometric content and fine-grained geometric visual perception, which may help better characterize the geometric perception capabilities of current VLMs.

Critical weaknesses
  1. The proposed formulation of “fine-grained geometric visual perception” is broader than what the actual solution and dataset support. In practice, FGGVP-22K mainly consists of coordinate prediction for predefined geometric primitives, query-conditioned localization of specified components, and analytic construction of a small set of relation-defined points such as midpoints, intersections, and perpendicular feet. These tasks are much narrower than the claimed notion of general fine-grained geometric visual perception and largely resemble specialized geometric grounding and coordinate prediction.
  2. It is difficult to understand what the figures are intended to convey, particularly Fig. 2. Furthermore, the aspect ratios are unreasonable, and the legends and fonts appear stretched. The authors need to carefully check the quality of the figures and ensure that the images accurately convey the intended ideas.
  3. The manuscript appears to deviate substantially from the standard ICLR formatting template, including the font problem. This should be corrected to ensure compliance with the submission guidelines.
Questions for authors
  1. How do the authors justify framing FGGVP as a general fine-grained geometric perception problem, rather than a narrower form of geometric grounding and coordinate prediction?

  2. Could the authors clarify the intended message of Fig. 2 and revise the figures to improve readability, aspect ratio, and font/legend quality?

  3. Could the authors revise the manuscript to fully comply with the standard ICLR formatting template, particularly the font and overall typesetting?

Reviewer_iHdJOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents the problem of fine-grained geometric visual perception (FGGVP), that deals with recovering precise positions, endpoints, boundaries and continuous trajectories. The authors suggest to solve the problem by constructing a dataset called FGGVP-22K containing 20K synthetic and 1.5K real samples that cover explicit primitive localization, semantically specified components, and derived targets determined by geometric relations. Using this dataset, the authors fine-tune a pretrained Qwen3-VL-4B with standard language loss with a few additional loss terms. The fine-tuned model shows better performance compared to the pretrained baselines on the presented task.

Critical strengths

The paper formulates the problem of FGGVP and shows that when a VLM is tuned on this downstream task, it outperforms pretreained VLMs.

Critical weaknesses
  1. The training data and downstream task are not clear. Does the FGGVP-22K dataset consists of only geometric shapes, lines, and points, and not realistic or natural images? If so, it is not surprising that the pretrained baselines would not perform well, as they werer probably mostly trained on real-word images and reasoning tasks. Visual or concrete examples from the constructed dataset (image, query, and target) in the main paper would help.

  2. The experiments are not compelling. The baselines do not include a VLM fine-tuned on the full proposed dataset without the loss and design changes. The “+ synthetic data” setting in Table 4 is synthetic-only and already uses the auxiliary losses, so it does not isolate the contribution of the data. This is especially important for the proposed problem which requires analyzing OOD data: the baselines were trained on mostly natural images, yet they are evaluated on a downstream task of geometric objects analysis. A fair baseline would be a model that was tuned on data from the evaluated domain. I did not find the presented ablation convincing. A useful ablation would compare a VLM fine-tuned on the entire new dataset with the additional losses (the presented approach) to a VLM that was fine-tuned without each of the added loss terms. This ablation will allow to evaluate each loss term on its own, and would provide a fair baseline that treats the downstream task as a non-OOD objective.

Both of the weaknesses stated above come from my main concern, which is that the model was evaluated on a downstream task that is inherently OOD for the pretrained baseline. The authors then show that when VLM is tuned on this new downstream task, its performance improves. While the claim is supported by the reported experiments, the novelty is limited.

Questions for authors
  1. How would a baseline fine-tuned on the proposed dataset, without the additional loss terms, perform?
Reviewer_Z193OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies fine-grained geometric visual perception: recovering precise coordinates for visible primitives, semantically specified components, and geometrically derived targets such as perpendicular feet and intersections. It introduces FGGVP-22K, comprising approximately 20,000 synthetic and 1,500 real samples. The method fine-tunes Qwen3-VL-4B using a discrete coordinate representation and a combination of autoregressive cross-entropy, ordinal-coordinate, expected pixel-distance, and coordinate-point CVaR losses. Evaluation covers synthetic data, Geo-Code-derived Image2Code data, and PGDP5K, together with downstream geometric reconstruction. The paper reports substantial localization improvements over the base model and API baselines, and improved reconstruction relative to Geo-Code and GeoCodeLM.

Critical strengths
  1. A useful task formulation with structured supervision. The dataset goes beyond recognizing geometric categories by supervising precise locations and unmarked, relation-defined targets. Analytically generated coordinates provide a principled source of supervision, and the shared coordinate interface connects perception to subsequent geometric computation.

  2. Substantial reported improvements with downstream relevance. On PGDP5K, the proposed model reports errors of 4.467/2.867/3.561 pixels across the three task categories, compared with 88.06/90.30/82.35 for the base model. The reconstruction results are also promising: Chamfer Distance improves from 9.836 to 3.640 and SSIM from 0.8520 to 0.9560 relative to Geo-Code. These results suggest practical value beyond matching the training output format.

Critical weaknesses
  1. The contribution of the proposed training components is not established. Section 3.4 introduces a pretrained geometric adapter without specifying its training data or procedure. There is no matched cross-entropy-only baseline or ablation of the three auxiliary losses. Moreover, Table 4 shows an unusually large improvement from synthetic-only to complete training—for example, 99.80 to 1.331 pixels on synthetic derived targets—without establishing whether initialization, training budget, and other settings are identical. Consequently, the respective contributions of the dataset, adapter, and objective remain unclear.

  2. The evidence for generalization and downstream transfer is insufficiently documented. Both the real training data and the Image2Code test set originate from Geo-Code, but image-level separation is not specified. The construction of the three evaluation categories on external datasets, point-correspondence rules, and treatment of invalid predictions are also unclear. Finally, the conversion from query-conditioned coordinate outputs to complete Image2Code reconstructions is not described sufficiently to assess whether the comparison uses equivalent models, tools, and refinement budgets.

Questions for authors
  1. Data provenance and independence: What data were used to pretrain the geometric adapter, and how are the approximately 1,500 real training samples separated from the 1,172 Image2Code test samples? The original Geo-Code paper describes a curated corpus of 1,510 image-code pairs, making the relationship between these subsets particularly important. Please clarify whether separation is enforced at the underlying image level, including across different queries or renderings of the same diagram.

  2. Attribution of improvements: Can you provide a cross-entropy-only baseline using the same data, initialization, and training budget, together with ablations of the auxiliary losses? For Table 4, are the synthetic-only and complete settings identical apart from the real data, and what explains the reduction from 99.80 to 1.331 pixels on synthetic derived targets?

  3. Objective specification: What are the exact definitions of the ordinal and expected pixel-distance losses, the loss weights, and the CVaR risk level? How are differentiable point errors constructed from the autoregressive high/low coordinate-token distributions? Please also provide the referenced appendix containing the training and replay settings.

  4. Evaluation and reconstruction protocol: How are target-point ordering, missing or malformed outputs, and coordinate normalization handled consistently across models? For Table 3, how are the predicted coordinates converted into complete executable reconstructions, and which additional models, tools, or iterative refinement steps are used?

1 comment
Reviewer_Z193OpenReview ↗

The authors did not provide a rebuttal

The authors did not provide a rebuttal. I maintain my opinion of rejecting the submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.