acceptodds
Withdrawn from ICLR 2027

UniPoly: Unified End-to-End pipeline for Raster2Vector Tasks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatial vectorization provides structured representations for urban modeling, digital twins, indoor navigation, and Geographic Information Systems (GIS). Existing methods often rely on task-specific architectures or multi-stage pipelines, while evaluation protocols remain fragmented across spatial domains. We propose UniPoly, a unified end-to-end framework for raster-to-vector (R2V) spatial reconstruction that represents diverse spatial objects as polygons. UniPoly adopts a shared polygon-based reconstruction paradigm with configurable components and introduces Poly Relational Attention (PRA) to model interactions within and across polygons. We further establish a unified evaluation protocol for both multi-stage and end-to-end methods, assessing region accuracy, geometric complexity, boundary fidelity, and vector fidelity. Extensive experiments on four indoor and outdoor benchmarks for floorplan, building, farmland, and road reconstruction show that UniPoly achieves the best or second-best performance on 31 of 32 evaluated metrics, with eight metrics per benchmark, while maintaining competitive inference efficiency. These results demonstrate the effectiveness of UniPoly across diverse spatial domains. Code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unified Pipeline, Raster-to-Vector, Spatial Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CgrvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UniPoly, a unified end-to-end raster-to-vector framework that reconstructs a set of polygons from diverse spatial inputs. The key architectural idea is a shared polygon-query decoder with configurable attention, including a Poly Relational Attention (PRA) module that decomposes query interactions into intra-polygon and inter-polygon attention, plus geometry-aware multi-level optimization with differentiable rasterization. The authors also assemble a unified evaluation protocol spanning region accuracy, geometric complexity, boundary fidelity, and vector fidelity, and report strong results across four representative benchmarks.

Critical strengths
  1. Strong performance across diverse R2V tasks. UniPoly provides a unified polygon-based reconstruction framework and achieves competitive results across four spatial domains. The comprehensive evaluation protocol further strengthens its practical value.
Critical weaknesses
  1. Insufficient validation of the unification claim. The four tasks are trained independently with different configurations. Without joint-training or cross-domain experiments, the benefits of unification beyond architectural reuse remain unclear.
  2. Limited comparative validation of PRA. Although component-wise ablations are provided, comparisons with existing factorized or axial-style attention mechanisms are missing. The rationale for aligning edge queries across different polygons also remains insufficiently justified.
Questions for authors
  1. Could the authors further clarify the benefits of the unified framework beyond architectural reuse? Would joint-training or cross-domain transfer experiments help substantiate the unification claim?
  2. What motivates the alignment of corresponding edge queries across different polygons in PRA? How does PRA differ from existing factorized or axial-style attention mechanisms, beyond the component-wise ablations already provided?
Reviewer_8LShOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes a method to convert rasterized images into 2D polygons. Compared to prior work, the method emphasises generality across tasks/datasets, and proposes a new metric. The model is a standard encoder-decoder Transformer. The key technical contribution seems to be a novel attention mechanism called Polygon Relational Attention (PRA) that allows for attention between and within predicted polygons. The loss follows the literature. The method is compared to a large number of baselines and evaluated on multiple different datasets/tasks. The proposed method outperforms or at least performs comparable to most baselines and datasets.

Critical strengths

Strong performance across tasks and datasets using a single model.

Critical weaknesses

The paper reads well, it’s a strong performing method -- the key issue is that I do not know what to take away from this paper: Table 1 shows dataset statistics, Table 2 shows training configurations, Table 3 shows that the proposed method outperforms most other methods - and that’s it. There are no analysis or ablation studies so I do not know how I could use this paper to improve my own works. It is a well engineered method, but the paper seems to provide little additional scientific insights.

Actually, there is an ablation study in Table 4 in the appendix (which seems odd, why hide it there?). This study indeed analyzes the main technical contribution of the paper (PRA the attention mechanism), its effect seems somewhat helpful depending on the metric. However, after reading the implementation details (l.320-322) it appears that PRA (main technical contribution) is not used for indoor floorplan prediction - in that case, how does this method actually differ from RoomFormer, which already proposed what is called here GlobalSelfAttention? Indeed Figure 2 in this paper, and Figure 2 in RoomFormer look pretty much the same. Overall, the paper does not emphasize much it’s technical contributions, in particular the related work sections mentions some rather high level limitations of existing work, but does not explain how these limitations are addressed by this work. In summary, this method performs much stronger than a baseline like RoomFormer, but is remains unclear why that is the case. Is is just because this work uses a stronger feature backbone?

Questions for authors

How does the proposed method differ from RoomFormer (beyond technical details, like feature backbone)? To me, the model seems exactly the same, given that PRA is not used for indoor floorplan prediction.

It seems unusual that ALB is just mentioned as a side-note, in my opinion this could be presented a bit more prominently in the paper. How important is this part for the good performance?

Reviewer_6aMEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniPoly, a unified raster-to-vector reconstruction framework for indoor floorplans, building footprints, farmland parcels, and roads. The proposed method develops Poly Relational Attention (PRA) to model intra- and inter-polygon relationships, which is similar to cross attention. Besides, this paper claims that an eight-metric evaluation protocol covering region accuracy, geometric complexity, boundary fidelity, and vector fidelity. The experiments report competitive performance across four tasks and benchmarks. Overall, the work shows extensive experiments, however, the novelty is not very strong (sequence prediction for polygon generation) for an ICLR paper. The metrics are already widely used in building outline extraction. As a result, I rate this paper as weak reject.

Critical strengths
  • The paper addresses an important problem: spatial vector reconstruction methods are fragmented across application domains. This paper shows vectorization capability on 4 different tasks.

  • On different tasks, this paper shows competing results compared with many recently published papers. The reported results are promising, particularly for floorplan reconstruction and several boundary and corner metrics.

  • In appendix, this paper presents experiment details, which is helpful to reproduce the experiments.

  • In appendix, this paper shows many good results, compared with the baseline models.

Critical weaknesses
  • It's called an unified model, however, the models are trained separately. A "real" unified model is suppsed to be a foundation model like SAM.

  • Lack of important recent papers [a, b], which directly generate polygons using LLM.

  • The proposed evaluation protocol is not new. Many metrics are widely used, such as PoLiS.

  • Lack of comparison on recent building outline extraction methods [b, c]. The result on WHU-building is not better than recent papers [b, c].

  • Eq 1 is not informative enough. How are edge, cos, region losses defined?

[a] MARS - A Foundational Map Auto-Regressor, ICLR, 2026. [b] Rethinking language models for building outline extraction from remote sensing imagery, CVPRW, 2026. [c] Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery, WACV, 2025.

Questions for authors

How do you prepare the GT as a sequence from all the key points?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.