acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Point-Image Fusion with Context Enhancement for Panoptic Symbol Spotting in CAD Drawings

Abstract

Panoptic symbol spotting in computer-aided design (CAD) drawings jointly identifies countable object instances and assigns semantic labels to all graphical primitives, including those representing uncountable stuff. Reliable recognition depends on both the geometry of individual primitives and the spatial arrangement of surrounding strokes, with these cues encoded differently in vector and raster representations. We propose a point–image fusion framework that combines bidirectional backbone interaction with local context enhancement before primitive fusion, bringing complementary visual context overlooked by vector-only methods into primitive representation learning. Specifically, we introduce Hierarchical Point-Image Fusion (HPIF) to enable spatially aligned exchange of local geometric and visual cues, enriching point features with multiscale visual context from the image branch and image features with geometric information from vector primitives. To strengthen each primitive's representation with surrounding geometric and visual evidence, our Primitive-Oriented Context Enhancement (POCE) module combines radius-bounded attention over same-layer primitives with regular and shifted window attention on image features, further improving panoptic recognition. Extensive experiments demonstrate that our framework outperforms previous methods on FloorPlanCAD and ArchCAD-Public, achieving state-of-the-art panoptic quality (PQ) of 92.7 and 90.6, respectively. Code is available at http://anonymous.4open.science/r/code-BED2.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
CAD Panoptic Symbol Spotting, panoptic segmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_x5QXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a dual-branch framework for panoptic symbol spotting in CAD drawings. It introduces Hierarchical Point–Image Fusion (HPIF) for bidirectional feature interaction between point and image backbones, and Primitive-Oriented Context Enhancement (POCE) to aggregate local geometric and visual context. Experiments on FloorPlanCAD and ArchCAD-Public achieve 92.7 and 90.6 PQ, outperforming reported baselines. Ablation studies demonstrate the effectiveness of the proposed components.

Critical strengths
  1. The method consistently outperforms existing approaches on two benchmarks, improving PQ over VecFormer by 1.6 points on FloorPlanCAD (with priors) and 3.8 points on ArchCAD-Public. These gains demonstrate its practical effectiveness.

  2. HPIF and POCE are well motivated for combining primitive geometry with visual context. The component and directional ablations provide evidence that hierarchical interaction and local context enhancement contribute to the improvements.

Critical weaknesses
  1. The framework primarily combines established feature alignment, bidirectional fusion, and local/window attention mechanisms. Moreover, bidirectional HPIF improves PQ by only 0.17 points over I2P-only fusion (Table 4(b)), leaving the necessity of the additional interaction direction insufficiently justified.

  2. The architecture uses two backbones, eight HPIF modules, and multi-stage training, but provides no parameter counts, FLOPs, inference latency, or memory comparisons. Results from repeated runs are also missing, making it difficult to assess the cost-effectiveness and statistical reliability of the reported gains.

Questions for authors
  1. Can the authors provide compute-matched comparisons between late fusion, I2P-only fusion, and bidirectional HPIF, together with results across multiple random seeds, particularly given the relatively small additional gain from P2I?

  2. What are the parameter counts, FLOPs, inference latency, and GPU memory requirements compared with VecFormer and DPSS? Are the performance improvements proportional to the additional computational cost?

Reviewer_fWLCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a point-image fusion framework for panoramic symbol recognition in CAD drawings. The point branch adopts PTv3, while the image branch employs UHRNet. Bidirectional feature interaction is performed across multiple stages via HPIF, followed by the aggregation of local primitives and image context through POCE. Finally, the framework predicts primitive semantic labels and instance grouping.

Critical strengths

Critical Strengths

(1) The motivation is clear, and the design is well-aligned with the characteristics of CAD drawings. (2) The performance improvements are consistent. (3) A relatively systematic module-wise ablation has been presented.

Critical weaknesses

Critical Weaknesses (1) The methodological contribution could be more clearly distinguished from established fusion approaches (2) The improvements brought by certain components appear relatively small, and the paper would benefit from including statistical uncertainty estimates to validate their significance. (3) The computational costs and the comparability with external baselines are not sufficiently characterized. (4) The primary ablation studies fail to establish the effectiveness of individual components in the absence of layer priors. (5) The analysis offers limited insights into instance grouping and failure modes (6) The reproducibility details should also be clarified

Questions for authors

Questions For Authors

  1. The methodological contribution could be more clearly distinguished from established fusion approaches

HPIF integrates bilinear sampling, feature splatting, gated residual updates, and multi-stage interaction, while POCE combines local neighborhood attention with both regular and shifted window-based image attention. These mechanisms have substantial precedents in the literature, including the related works discussed in the paper, such as DeepInteraction, ViT-CoMer, and Swin Transformer.

At present, the core contribution seems to rest on the integration and adaptation of these modules for CAD primitive representations. To enhance the paper's impact, it would be beneficial to pinpoint the specific shortcomings of prior fusion mechanisms within this domain and illustrate how the proposed architecture overcomes these challenges.

  1. The improvements brought by certain components appear relatively small, and the paper would benefit from including statistical uncertainty estimates to validate their significance.

In Table 4(b), with POCE retained, I2P alone achieves a PQ of 92.57, whereas bidirectional interaction yields 92.74—a marginal gain of 0.17 points. Similarly, in Table 4(c), PCT alone achieves 92.66 PQ, while combining PCT and ICT results in 92.74, representing an additional gain of only 0.08 points. Furthermore, the performance difference between three-point and five-point image sampling is merely 0.09 points.

While these results warrant further investigation, the paper fails to report means, standard deviations, or confidence intervals from repeated runs. Consequently, the statistical reliability of these marginal differences remains uncertain.

  1. The computational costs and the comparability with external baselines are not sufficiently characterized.

The framework employs a dual-backbone architecture, eight HPIF modules, and additional context attention, along with a multi-stage training procedure that involves independent branch training, joint optimization, and subsequent context-module training. However, the paper fails to report key efficiency metrics, such as parameter counts, FLOPs, inference latency, peak memory usage, or training GPU-hours.

The FloorPlanCAD baseline results are directly adopted from prior literature. While retraining VecFormer on ArchCAD-Public is a positive step, merely stating that it was trained to convergence is insufficient to ensure full comparability in terms of pretraining, optimization budget, and tuning effort. Although the internal ablations help assess individual components, an external accuracy–efficiency comparison remains necessary.

  1. The primary ablation studies fail to establish the effectiveness of individual components in the absence of layer priors.

Table 4 presents all component ablations on FloorPlanCAD with layer priors, which contrasts with the paper's emphasis on practical settings where such metadata is unavailable. Since PCT explicitly restricts neighborhoods to primitives within the same layer, its performance in a single-layer, no-prior setting is particularly critical to evaluate.

I recommend conducting the principal ablations for HPIF, PCT, and ICT on FloorPlanCAD without priors, as well as on ArchCAD-Public. These experiments would also help elucidate which components are responsible for the more substantial gains observed on ArchCAD-Public.

Additionally, the point representation incorporates a normalized layer identifier as its third coordinate. Since layer identifiers are inherently categorical, their numerical ordering does not necessarily convey geometric meaning. The paper should clarify the identifier assignment protocol and evaluate the model's robustness to permutations that preserve layer membership while altering the numerical values.

  1. The analysis offers limited insights into instance grouping and failure modes

Figure 2 and the supplementary visualizations primarily depict primitive-level semantic coloring. While these examples illustrate classification differences, they do not directly reveal whether adjacent primitives of the same category are correctly separated into distinct instances.

Given that the task is panoptic symbol spotting, I recommend incorporating instance-level visualizations, along with a breakdown of Recognition Quality (RQ) and Segmentation Quality (SQ). Additionally, results stratified by category or symbol density would provide deeper insights. It would be particularly informative to analyze challenging cases involving nearby repeated symbols, overlapping strokes, and small objects, as well as to include representative failure cases such as instance merging, splitting, and missed detections.

While the paper appropriately identifies complete-drawing inference as future work, it would still be valuable to report performance and memory usage as a function of primitive count within the current cropped-region setting. Such data would provide useful evidence regarding the method's scalability.

  1. The reproducibility details should also be clarified

Several implementation details require clarification to ensure reproducibility, including raster resolution, stroke width, line-segment sampling rules, input geometric attributes, sources of pretrained weights, ICT window size, and the set of trainable components at each optimization stage. While the appendix specifies default PCT neighborhood parameters, these additional configuration choices are equally critical.

The highest-priority additions include repeated runs for key ablations, component analysis without priors, and accuracy–efficiency comparisons with strong baselines. These experiments would substantially strengthen the evidence for the necessity of bidirectional fusion and POCE, while helping to establish the paper’s contribution beyond a mere combination of existing mechanisms.

https://iclr.cc/Conferences/2027/AuthorGuidelines Recommended: Reproducibility statement It is important that the work published in ICLR is reproducible. Authors are strongly encouraged to include a paragraph-long Reproducibility Statement at the end of the main text (before references) to discuss the efforts that have been made to ensure reproducibility.

Reviewer_wBPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a point-image fusion framework for panoptic symbol spotting in CAD drawings. The proposed method combines geometric features from CAD primitives with visual features from raster images through hierarchical bidirectional feature interaction (HPIF), followed by a context enhancement module (POCE) to capture relationships between nearby primitives and surrounding image regions. Experiments on FloorPlanCAD and ArchCAD-Public show that the method outperforms existing approaches, achieving PQ scores of 92.7 and 90.6, respectively. The results suggest that combining geometric and visual information can improve recognition performance, although the added complexity and the method’s generalization to full-scale CAD drawings remain worth investigating.

Critical strengths
  1. The paper addresses a practical problem in CAD drawing recognition and achieves strong results on both FloorPlanCAD and ArchCAD-Public.
  2. The HPIF module enables bidirectional interaction between point and image features during feature extraction, making better use of the complementary geometric and visual information.
  3. The ablation studies show that both HPIF and POCE contribute to performance improvements. In particular, HPIF provides a clear gain, while POCE offers an additional improvement through local context modeling.
Critical weaknesses
  1. The core novelty is not sufficiently clear. HPIF and POCE mainly build on existing feature fusion and attention mechanisms, and the paper does not fully explain how these designs differ fundamentally from previous methods. The contribution appears to lie more in combining existing techniques than in introducing substantially new mechanisms.
  2. The gains from some components are relatively small. POCE improves PQ by only 0.37 percentage points, and the paper does not report results across multiple runs or provide standard deviations. It is therefore difficult to determine whether this improvement is consistent and reliable.
  3. The evaluation does not fully support claims about practical applicability. Experiments are limited to cropped CAD regions, with no direct evaluation on full-scale drawings. The lack of comparisons in inference speed, memory usage, and computational cost also makes it difficult to assess whether the accuracy gains justify the added complexity.
Questions for authors
  1. How does HPIF differ from existing point-image fusion methods in terms of its core mechanism, and what evidence shows that bidirectional feature interaction is necessary rather than simple feature concatenation or one-way fusion?
  2. POCE improves PQ by only 0.37 percentage points. Have the authors tested whether this improvement remains consistent across multiple runs, and how sensitive is the result to different settings of the context enhancement module?
  3. Since the experiments are conducted on cropped CAD regions, how well does the proposed method scale to full-size CAD drawings with a much larger number of primitives, particularly in terms of inference time and memory consumption?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.