acceptodds
Under review as a conference paper at ICLR 2027

Prompt-Switchable Topological Vectorization: Structured Geometry Decoding from a Frozen Segmentation Foundation Model

Abstract

Downstream geospatial applications often require segmentation outputs to be represented as polygons satisfying hard geometric and topological constraints: one polygon per instance, no overlapping interiors, and exact shared boundaries between adjacent polygons. Converting masks to polygons is usually left to post-processing, which makes topology something to repair rather than something to guarantee. We move these constraints into the decoder: a frozen promptable segmentation backbone is adapted with LoRA and a geometry head on the same features, M trainable parameters versus M frozen, and the predicted geometry fields are decoded into a planar graph whose faces satisfy these constraints by construction. Our experiments reveal two broader findings. Instance metrics are close to blind to structural decodability: dropping the geometric supervision costs half a point of micro F1 while raising uncovered instances from 11.3% to 96.7%. And an open vocabulary at the prompt does not reach the output: switching the prompt alone gives micro F1, whereas ten annotated images give . On three cropland domains, the topological output outperforms all evaluated baselines, with a -point gain on the one genuine zero-shot transfer, while returning ten to sixteen times the shared boundary of a contour pipeline, and still seven to nine times after that pipeline's output has been through a standard GIS repair.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
topological vectorization, structured geometry decoding, prompt-switchable segmentation, segmentation foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zZkLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the failure of independent mask contouring to produce valid cadastral polygons, which routinely leaves overlaps and sliver gaps along adjacent boundaries. The authors attach a lightweight geometry prediction head and LoRA adapters to a frozen SAM 3 backbone. During inference, the predicted boundaries, corner heatmaps, and sub-pixel offsets are vectorized into a planar graph. Polygons are extracted as the faces of this graph and labeled by majority overlap with the segmentation masks, ensuring adjacent parcels share identical arcs by planar construction rather than post-hoc geometric repair.
Contribution Replaces loss-based penalties and post-processing heuristics with planar graph face decoding, enforcing non-overlap and shared boundaries by construction. Uncovers an evaluation limitation in standard region overlap metrics, demonstrating that F1 scores remain largely unchanged even when underlying feature representations become structurally undecodable. Clarifies the practical limits of foundation model prompting for structured geometries, showing that zero-shot text prompting fails to transfer edge topology across distinct object classes without low-shot retraining of the geometric features.
Validation The method was evaluated on three agricultural parcel datasets and one building dataset. Evaluation focused on both instance matching and downstream topological validity, measuring polygon overlaps, gap frequency, and shared boundary continuity against dedicated cropland and building baselines as well as standard GIS cleaning routines.

Critical strengths

Topological decoding by construction: Instead of post-hoc GIS fixes or loss penalties, the decoder builds a planar graph directly from predicted edge and corner fields, ensuring adjacent faces share the same simplified segments. The setup eliminates boundary slivers, and while the residual fallback reintroduces a small amount of overlap (~0.004), it remains well below contour baselines. The prompt-switching experiments are also grounded realistically: zero-shot transfer on buildings drops to 0.08 F1, but adapts to 0.817 with just 10 annotated images.
Limitations of F1 for structural evaluation: The ablation in Table 3 shows that standard region metrics fail to measure polygon decodability. Dropping the geometric branch during LoRA adaptation changes micro F1 by only 0.54 points, yet uncovered instances jump from 11.3% to 96.7%. This clearly shows that instance F1 alone is insufficient for evaluating representation quality in vectorization.

Critical weaknesses

Prompt switching fails on buildings zero-shot (0.08 micro F1), so the model really relies on 10-shot PEFT to do anything useful. More importantly, testing only cropland and buildings avoids the real problem. Cropland is pure tiling and buildings are isolated islands; the paper never evaluates whether this promptable pipeline can handle mixed maps where parcels, roads, and structures meet at shared nodes.
The main pitch is replacing post-hoc repairs with planar graph construction. But the planar graph misses up to 21% of instances, forcing the authors to glue contour polygons back in via the residual fallback. This puts interior overlaps back into the delivered layer and voids the exact topological guarantee of eq. (2).
The network just stacks LoRA and a CenterNet head onto a frozen SAM 3. The topological decoding contains no learned graph operators—it is standard morphological skeletonization, Douglas-Peucker, and GIS polygonization tied together with heuristic thresholds.
Baselines are evaluated out-of-the-box without adaptation, while the proposed model is trained on two domains, fine-tuned on the third, and has its post-processing thresholds searched directly on the AI4Boundaries evaluation split.

Questions for authors

I find the downstream GIS evaluation and the paper's transparency compelling. Addressing the following points directly in the rebuttal would help me reconsider my score: Evaluation-split tuning: In Appendix D, the post-processing grid search ran directly on the AI4Boundaries evaluation images. What is the performance if thresholds are tuned on a held-out training split instead? Alternatively, how do the baselines perform if given an equivalent post-processing search on this split?
Cross-category and road-network consistency: Experiments are confined to isolated cropland tiling or standalone buildings. In operational mapping, parcels, road networks, and building footprints meet at shared junctions and linear boundaries. If prompts for fields, roads, and structures are executed on the same scene, can the planar graph preserve topological consistency across layers, or do cross-category slivers and overlaps inevitably reappear?
Fallback proportion: Across the test splits, what exact percentage of delivered polygons comes from the planar graph versus the residual fallback? How should users distinguish polygons that actually satisfy the Eq. (2) guarantee from those that do not?

Reviewer_2sXQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

They propose to predict boundary and corner masks in addition to regions. They skeletonize and snap the edges and image boundary to create a partition of the image into faces. Each face is labeled according to the predicted mask that most overlaps it. When they fail to partition the image domain they fall back to polygonized region masks. They call this geometry decoding -- vs repair. Also, their model is LORA-adapted SAM with additional point+corner heads.

Critical strengths

They compare to and seem to outperform other methods on the same databases; the method is pretty easy to understand.

Critical weaknesses

It is unclear what the difference is between "geometry decoding" vs "repair" is. Unsure why micro-F1 (or micro in general) is the right metric. It is unclear what role prompting has in this paper.

Their "fallbacks" make sense -- but they also undercut their point a bit.

A key aspect of their contribution seems to be related to segmenting features that should peartition the image plane -- that is, without gaps between them. Examples are parcels, fields/crop boundaries. These tend to touch and (perhaps) should not have gaps between them (although I think they sometimes do). Then they show how it can be adapted to buildings. I can see how the additional corner/edge targets would help segment better buildings, but the concept of a shared boundary between buildings—it doesn't happen often (It certainly does in downtown cities, of course). I would like more explanation of how/why this approach is also applies to building segmentation.

There are several places in the paper that I still cannot decipher what the authors are trying to say -- they use sophisticated words, but strung together in a way I cannot decipher. If they are using LLM assistance, (as I expect all wuthors to do nowadays), I would encourage them to push for "plain old English."

Post Rebuttal: The authors have addressed several concerns, but do not change my assessment of the overall significance. I will stick with the "weak accept/3" rating.

Questions for authors

You stress at many times the use of "micro-F1". Is there a reason that this is the correct metric to use? When I hear it, I worry about class-imbalance effects.

Where do you draw the line between "geometry decoding" vs "repair"? Is it the addition of additional topological targets (corners, boundaries) that make this a geometry-decoding vs. repair?

What types of prompt might be used for the parcel segmentation tasks? It seems like you found that they had a negligable effect.

What does this mean?

"The topological path buys shared boundary and pays in F1. Across foru domains it returns 10.2 and 15.8 times the shared boundary length of the ccontourpath on the same images."

Reviewer_BBpCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies prompt-conditioned extraction of vector polygons from remote-sensing imagery, emphasizing non-overlapping polygons and shared boundaries between adjacent instances. It adapts SAM 3 using LoRA and a geometry head that predicts edges, corners, and offsets, with approximately 7.1 million trainable parameters. A decoder constructs a planar graph, simplifies shared segments, polygonizes the graph, and assigns faces to predicted instance masks. Contour-based fallbacks recover instances that the graph fails to represent. Experiments cover three cropland domains and building adaptation. The paper reports improved cropland F1 over the evaluated baselines and substantially higher shared-boundary rates than its own contour decoder. It also shows that similar instance F1 scores can accompany very different graph coverage, and that building extraction improves substantially after adaptation on ten annotated images. The application and representation choice are compelling, but major evidentiary gaps remain concerning evaluation independence, structural correctness, comparison fairness, and the specific benefits of geometric coupling and prompting.

Critical strengths
  1. The paper addresses an important distinction between segmentation accuracy and usable vector structure.

  2. The comparison between decoders using identical model weights is informative.

  3. The PASTIS result provides meaningful evidence of cross-domain transfer.

  4. The manuscript is explicit about unfavorable results and limitations.

Critical weaknesses
  1. Reported evaluation results are affected by parameter selection on the same images. Appendix D reports selecting 192 post-processing configurations and 26 topological configurations using the same 351 AI4Boundaries images used for evaluation. Building parameters were selected on a 50-image subset of the reported WHU evaluation split. Consequently, these results are not independent estimates after model selection. This particularly affects the interpretation of “ten annotated images”: ten images are used for adaptation, but additional labeled evaluation images inform post-processing selection. Disclosure is valuable, but does not remove selection bias. Parameters should be selected on a separate development split, with the total labeled-data budget stated explicitly. PASTIS remains the cleaner transfer result because its configuration was inherited.

  2. The baseline comparisons do not adequately isolate the proposed method’s contribution. The proposed model receives training on AI4Boundaries and iFLYTEK, while all specialist baselines use released weights without adaptation. These comparisons establish performance against available checkpoints, but cannot distinguish architectural benefits from task-specific training. The open-vocabulary comparison also has a particularly consequential bottleneck: SegEarth-OV3’s fused semantic raster is converted into connected components, which merges touching parcels. Its underlying SAM 3 already produces discrete instance predictions before semantic aggregation. A direct SAM 3 instance-mask baseline, followed by the same contour processing, would be a closer control. The relevant intermediate predictions are explicitly described in the SegEarth-OV3 method.

  3. The principal structural metric measures boundary proximity, not correct topology. — Major; demonstrated measurement gap Equation 4 measures how much predicted boundary lies within one pixel of another predicted boundary. It neither checks ground-truth adjacency nor verifies that neighboring polygons reference the same stored arc. Incorrect subdivisions or artificial contacts can increase this quantity. This is especially problematic when the increased shared-boundary rate on largely detached buildings is presented as a transferable benefit. More shared boundary is not inherently desirable there. The downstream evidence is also mixed: on PASTIS, Topo has approximately the same total overlap-plus-gap count as Contour, and worse area error, 0.138 versus 0.100. Ground-truth adjacency precision/recall and shared-boundary localization accuracy are needed to establish that the structural changes are correct.

  4. The hard guarantees apply to a restricted intermediate output, rather than the complete delivered result. Planar faces provide a valid non-overlap construction. However, image-level contour fallbacks and appended residual polygons lie outside that construction. Delivered overlap reaches 0.0038 on AI4Boundaries and 0.0079 on iFLYTEK. Disabling residual fallback on AI4Boundaries leaves 11.3% of ground-truth instances uncovered. Furthermore, planar-face consistency does not guarantee one polygon per actual instance: a face can merge objects, and multiple faces can split one object. The paper acknowledges fallback overlap, so this is not an undisclosed flaw. The remaining issue is that the advertised application requirements exceed what the delivered system guarantees. Claims should consistently distinguish representation validity, instance correctness, and coverage.

  5. The ablation does not isolate the claimed benefit of geometry gradients flowing through LoRA. — Major; unresolved causal interpretation Table 3 compares joint training, LoRA-only training, and geometry-head-only training. Removing geometry supervision predictably damages an output head that depends on that supervision. Conversely, training the geometry head on an entirely frozen backbone does not isolate the effect of geometric gradients on an otherwise task-adapted backbone. A decisive control would train both heads and LoRA, but stop geometry-loss gradients at the shared features while retaining detection-loss gradients into LoRA. This would distinguish the benefit of geometric supervision from the benefit of coupling that supervision to adaptation. The authors should also clarify how image-level contour fallback contributes to each ablation’s F1; otherwise, stable F1 may partly reflect switching to a different output route.

  6. Few-shot adaptation is demonstrated more convincingly than prompt switchability. Building F1 increases from 0.0800 to 0.8173 after supervised adaptation on ten images. This demonstrates effective adaptation, but does not establish that text prompting is responsible for it. The paper does not show that one fixed adapted checkpoint supports both categories through prompt changes while retaining cropland performance, or compare against a fixed-prompt/category-conditioned control. The authors correctly withdraw an open-vocabulary claim. Nevertheless, the remaining “prompt-switchable” contribution needs operational evidence beyond a prompt interface and separately adapted checkpoints.

  7. The novelty relative to closely related shared-edge reconstruction is insufficiently differentiated. The paper cites ACPV-Net but offers little technical comparison. ACPV-Net, published at CVPR in June 2026, already reconstructs a common planar graph, decomposes it into anchor-bounded polylines, simplifies using predicted vertices, and extracts labeled faces with shared boundaries. These mechanisms overlap substantially with the broad structured-decoding contribution. See the published record and technical description, Section 4.2. Meaningful differences remain: this paper traces learned edge fields and uses a promptable, parameter-efficient backbone, whereas ACPV-Net starts from semantic transitions and uses diffusion-based vertex prediction. The paper should explain what these differences enable and provide a controlled comparison where feasible. The overlap does not establish that the complete method lacks novelty, but it narrows where that novelty can reside.

Questions for authors
  1. Can you report AI4Boundaries and WHU results using parameters selected on disjoint development data? How many labeled images are used in total for adaptation, checkpoint selection, and post-processing selection?

  2. What are the ground-truth shared-boundary rates and adjacency precision/recall? Does the increased WHU shared-boundary rate represent correct contacts or artificial connections?

  3. With both image-level and residual fallbacks disabled, what are F1, coverage, and overlap on every domain? Does the exported representation preserve shared arc identities so that editing one boundary actually updates both polygons?

  4. What happens when the geometry head remains supervised but its gradients are prevented from updating LoRA? How much of the LoRA-only ablation’s F1 comes from contour fallback?

  5. Can one fixed checkpoint extract both buildings and cropland by changing only the prompt? What happens with a fixed or incorrect prompt, and how much cropland performance is retained after building adaptation?

  6. How does direct SAM 3 instance output perform under the same vectorization protocol? What benefit remains against a specialist adapted on the same data, and how does your reconstruction differ empirically from ACPV-Net?

  7. Please specify training-set sizes, split identifiers, image resizing, PASTIS band/time selection, geometry-target construction, the early-stopping split, and the exact criterion for an “uncovered” instance. These details materially affect both the fixed pixel thresholds and the reported coverage results.

  8. Is topological decoding process differentiable? Or is it just a post-processing? Is the framework end2end?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for AC, SAC and PC

Thank you for handling our submission. To assist your assessment, we summarize the main concerns raised by the reviewers and the evidence added to address them. The reviews prompted further tests of model selection, baseline comparability, geometric-gradient coupling, and the distinction between structural guarantees and instance accuracy.

  1. Independent model selection and comparable baselines (BBpC, zZkL; Section 5, Table 4, Appendix F). We retrained using 3,224 training images and 792 development images. Development loss selects checkpoints; 161 AI4Boundaries development images select post-processing parameters, which are then fixed across all three cropland domains. Development and evaluation image IDs are disjoint, and PASTIS contributes no training or selection data. We added direct SAM3 instance masks with Contour decoding and supervised adaptation of BsiNet and ACPV-Net on the same training/development split. Both our Contour and Topo outputs outperform these three baselines across all three cropland domains. Topo achieves micro F1 of 0.6008, 0.4867, and 0.6037 on AI4Boundaries, PASTIS, and iFLYTEK, respectively.

  2. A controlled test of geometric-gradient coupling (BBpC, zZkL; Table 5, Appendix F). Following the proposed stop-gradient control, both branches remain supervised while only the geometry-loss gradient into LoRA is blocked. With identical initialization, seed, 25,792 updates, and decoder settings, retaining geometric gradients improves strict graph F1 by 1.36, 2.23, and 1.41 percentage points across the three domains. This supports the contribution of geometric gradients at a matched training budget. We also report development-selected checkpoints separately: stop-gradient has higher strict graph F1 under that selection procedure. The evidence therefore supports a fixed-budget effect, rather than uniform superiority under every checkpoint-selection setting.

  3. Structural correctness measured against ground truth (BBpC, zZkL, 2sXQ; Table 5, Appendix F). We added GT-referenced adjacency evaluation and shared-boundary localization, rather than relying only on proximity between predicted boundaries. Unmatched objects remain accounted for in global adjacency precision and recall. Using the same weights, strict graph achieves exact adjacency F1 of 0.1298, 0.2175, and 0.1995, compared with 0.0026, 0.0000, and 0.0071 for Contour. These results show improved recovery of annotated adjacency, while their absolute values also expose the remaining structural errors.

  4. Explicit separation of graph guarantees and fallback output (BBpC, zZkL, 2sXQ; Sections 3.4 and 5, Appendix F). Strict graph disables both image and residual fallback and returns only graph faces; Topo adds fallback polygons to recover instances. We report their instance accuracy separately and record polygon provenance. Strict graph has zero mean internal overlap across the four evaluated domains; fallback polygons in Topo lie outside this guarantee. Graph faces account for 81.32%–86.37% of cropland Topo polygons. We also distinguish predicted-instance assignment diagnostics from GT recall, and clarify that shared boundary coordinates do not implement persistent shared-edge identifiers or linked editing.

  5. A bounded category-transfer claim and complete annotation budget (BBpC, zZkL, 2sXQ; Section 5). We claim prompt-switchable geometric adaptation, rather than zero-shot open-vocabulary vectorization. WHU experiments use 10 adaptation images plus 50 independent development images: 60 labelled images in total. Across three seeds, Topo reaches F1 of 0.8253 ± 0.0154 and Contour 0.8715 ± 0.0024. Buildings test adaptation and instance geometry; contiguous parcels test shared-boundary recovery. Fixed-checkpoint prompt controls and multiclass joint decoding remain outside the demonstrated scope.

Together, these additions clarify what the method guarantees, what its learned geometry contributes, and where fallback or target geometry limits the conclusions. The controlled experiments and supporting implementation details are included in the revised manuscript in dark blue. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.