acceptodds
Withdrawn from ICLR 2027

CrossFreq-UHR: Context-Aligned Frequency Learning for Ultra-High-Resolution Object Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Sparse global–local detectors improve ultra-high-resolution object detection by allocating high-resolution computation to a limited number of informative regions. However, their global observations are mainly used for region selection and downstream decoding, leaving selected patches to form their multi-scale representations without location-specific scene context before aggregation. We refer to this limitation as the pre-aggregation representation bottleneck. To address it, we propose CrossFreq-UHR, a context-aware frequency learning framework for sparse UHR detection. CrossFreq-UHR maps global features to the coordinate system of each selected patch to provide spatially aligned scene context, and decomposes local features into a smoothed structural representation and a detailoriented residual. An asymmetric interaction mechanism conditions the structural path on aligned global context while independently refining patch-specific detail, introducing scene awareness without directly perturbing local fine-scale information. The enhanced representations are subsequently aggregated for query-based global–local detection. Experiments on SODA-A and STAR demonstrate that the proposed pre-aggregation conditioning consistently improves detection accuracy over the sparse global–local baseline with modest additional computation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Ultra-High-Resolution Object Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WLvHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CrossFreq-UHR, which adds a feature enhancement block to the UHR-DETR pipeline. The block samples global context according to each selected patch’s coordinates, separates local features into a smoothed component and a reconstruction residual, and processes them through different branches. Global context is introduced into the structural branch, while the detail branch is refined locally. Experiments are conducted on SODA-A and STAR.

Critical strengths
  1. The method addresses a practical problem: how to use global information to improve selected high-resolution patches without processing the entire image at full resolution.
  2. The reported results improve over the baseline on both datasets with limited additional inference cost. The matched-patch-budget and parameter-controlled ablations also provide evidence that the added block is useful.
Critical weaknesses
  1. My main concern is that the technical contribution is limited, and the experiments do not clearly support its central explanation. Most of the detection pipeline is inherited from UHR-DETR. The added block mainly combines coordinate-based feature sampling, smoothing and residual construction, and branch-specific feature refinement. More importantly, Table E shows that spatial alignment does not outperform the alternatives when each variant is trained separately. The shifted variant achieves 56.25 AP50 and 30.58 AP, compared with 56.08 and 29.47 for the aligned variant. Even spatial averaging achieves a higher AP of 30.29. Alignment helps large objects, but these results do not establish that it is the key missing connection described in the paper.
  2. The comparison with existing methods is incomplete, and several baseline differences need explanation. The UHR-DETR paper reports 58.0 AP50 for SparseFormer and 59.5 for CEASC on SODA-A, but neither appears in this paper’s SODA-A comparison, where CrossFreq-UHR reaches 56.1. These methods are slower, but including them would give a more complete picture of the accuracy–efficiency trade-off.
Questions for authors
  1. Why were SparseFormer and CEASC left out of the SODA-A comparison? Are their results in the UHR-DETR paper comparable with the setting used here? Including them, or explaining the differences in evaluation, would help assess the claimed accuracy–efficiency trade-off.
  2. What accounts for the different UHR-DETR baseline results: 34.9 versus 28.62 on STAR, and 53.3 versus 54.18 on SODA-A? Please clarify which results were reproduced and whether the baseline and proposed method used the same training and evaluation settings.
Reviewer_RLHSOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a pre-aggregation representation bottleneck in sparse global–local UHR detectors (UHR-DETR): the local features of the selected high-resolution patches are not conditioned on global context before multi-scale aggregation. To address this, it proposes the CrossFreq-UHR block, inserted at the P3 and P4 levels, with three components: SACM grid-samples the global feature at the patch coordinates to obtain spatially aligned context, EFD splits the local feature into a smoothed structural component and a residual, and ASDI injects the context only into the structural path while refining the detail path independently. The paper reports gains of +2.8 and +1.85 AP50 over UHR-DETR on SODA-A and STAR, respectively.

Critical strengths

S1. Clear problem definition with low additional cost. The observation that existing sparse detectors use global information only for routing and decoding is valid, and the additional cost of the proposed block is small (latency 0.647→0.676 s in Table 2; training memory of about 18 GB even for 8K inputs in Table C).

S2. Controlled experimental design and transparency. The experiments are carefully designed, including a parameter-matched component ablation (Table 6), a side-by-side comparison of same-checkpoint interventions and independently trained variants (Tables D and E), and comparisons at matched patch budgets (Table 5). Reporting results that are unfavorable to the main claim (Table E) is commendable.

S3. Consistent gains. All object-size groups (small/medium/large) improve on both datasets, and the method achieves the highest mAP among the compared methods on STAR (Table 1).

Critical weaknesses

W1. The core claim (spatial alignment) contradicts the paper's own experiments. The introduction (L82–85) states that replacing aligned context with averaged or shifted context "consistently degrades" performance and uses this as the rationale for the design. However, in Table E, where each variant is trained independently, averaged, shifted, and aligned context achieve 56.00, 56.25, and 56.08 AP50, respectively, showing almost no difference, and the shifted variant is also the best in AP, AP75, and AR. The same-checkpoint interventions in Fig. 5a feed different inputs at test time to a model trained with aligned context, so a performance drop is expected and does not demonstrate the necessity of spatial alignment. Therefore, the gains likely come from the additional computation of ASDI rather than from alignment, and the main paper overstates its conclusion by omitting this result.

W2. The actual gain and the contribution of each component are unclear. UHR-DETR in the main comparison (Table 2) achieves 53.3 AP50, whereas the baseline in the ablations achieves 54.18 (Tables 5 and 6), so the actual gain under controlled conditions is +1.9 rather than +2.8. In Table 6, SACM alone yields +0.87 and EFD alone +0.48, while the symmetric variant combining both yields only +0.61, lower than SACM alone; the components are not additive, and the overall gain depends heavily on ASDI. In addition, simply enlarging the patches from 512 to 1024 under the same local-pixel budget increases AP50 by +1.95 (Table F). Since this is a simple alternative that provides wider local context, a comparison with the baseline using larger patches is needed. Moreover, r=4 outperforms the default r=2 on most metrics (Table 3) without a justification for the default choice, and without repeated runs over seeds, it is difficult to judge the significance of differences of around 1 AP.

W3. Non-standard benchmark protocol. For SODA-A, images are enlarged 2× with bilinear interpolation and then zero-padded onto a 9600×9600 canvas (Appendix B.1), and the authors themselves state that this is not the native resolution. The UHR setting is thus artificially constructed, which favors sparse methods and disadvantages downsampling methods that resize the image to 1024. In addition, oriented boxes are converted to horizontal boxes on both datasets, and custom rules are applied to STAR, such as discarding images with either dimension ≤ 6000 and cropping to 8192, making the results incomparable with numbers in the literature. There is also no sliding-window baseline on SODA-A.

W4. Limited novelty and generality. SACM, which crops the global feature at the patch location and combines it with the local feature, is essentially the same as the global–local feature sharing in GLNet (Chen et al., 2019a), and low-/high-frequency separation based on smoothing/pooling is widely used in Octave Conv (Chen et al., 2019b), FreqFusion (Chen et al., 2024a), and others. EFD is a fixed operation without learnable parameters, and the authors themselves acknowledge in Appendix D.4 that it does not guarantee frequency separation, so "Frequency Learning" in the title is an overstatement. Furthermore, the method is applied to only one detector (UHR-DETR), which does not demonstrate its generality as a framework; both datasets are aerial imagery, and there is no comparison on gigapixel benchmarks such as PANDA [1] or with other sparse methods such as SparseFormer (Li et al., 2024).

Minor. Appendix A states that all experiments are conducted on a single L40 GPU, whereas Table 5 measures latency on two GPUs, and the measurement conditions for Table 2 are not specified, making efficiency numbers difficult to compare across tables. Typo: "decompositionco" in Fig. 5b.

References

[1] X. Wang et al., "PANDA: A Gigapixel-level Human-centric Video Dataset," CVPR 2020.

Questions for authors

Please check the weaknesses.

Reviewer_VAhzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CrossFreq-UHR, a context-aware frequency learning framework for sparse ultra-high-resolution object detection. Starting from a sparse global-local detector, the method uses Spatially Aligned Context Mapping to sample location-specific global context for each selected patch, Explicit Frequency Decomposition to construct a smoothed structural feature and a reconstruction residual, and Asymmetric Semantic-Detail Interaction to condition the structural path on global context while refining the local detail path independently. The enhanced P3 and P4 features are then passed to the inherited encoder and global-local decoder. On the authors' prepared SODA-A and STAR benchmarks, CrossFreq-UHR improves AP50 over UHR-DETR by 2.8 and 1.85 points, respectively, with moderate reported overhead.

Critical strengths
  1. The paper is clearly organized, and the pre-aggregation conditioning problem is easy to follow. Mapping global features into each selected patch before multi-scale aggregation is a reasonable way to introduce scene context without processing the full image at high resolution.
  2. The experimental section covers several relevant factors, including component contributions, context-mapping interventions, reconstruction scale, feature-pyramid placement, patch budget, routing coverage, memory scaling, and qualitative examples. The complete method consistently improves the sparse global-local baseline on both reported datasets.
Critical weaknesses
  1. The novelty of EFD is not sufficiently established. Its fixed Gaussian smoothing and residual construction are closely related to conventional low- and high-frequency decomposition. In particular, please explain how EFD differs from the Adaptive Spectrum Disentanglement module in SFDNet. The authors should clarify the conceptual and technical differences and provide a controlled comparison under the same detection framework.
  2. The core semantic and detail operators are not presented with sufficient clarity in the main paper. The semantic and detail paths are described only at a high level, while key operator structures and implementation choices are deferred to the appendix. Because these operators are central to ASDI, the main methodology section should provide a self-contained description of their architectures, inputs, outputs, and interaction process.
  3. The comparison with recent state-of-the-art approaches is incomplete. Relevant frequency-aware small-object detectors, including HS-FPN (AAAI2025) and SFDNet (ECCV2026), are not included in the main experiments. Comparisons with these methods, are recommended to establish the competitiveness of CrossFreq-UHR.
  4. The rationale for the semantic and detail operators is not adequately validated. The current component ablations evaluate the presence or absence of the overall modules, but do not show why the specific semantic interaction and detail-refinement designs are necessary. The authors should provide operator-level ablations and compare them with simpler alternatives to demonstrate that the proposed designs, rather than the added processing alone, account for the improvement.
Questions for authors
  1. Please explain how EFD differs from the frequency Decomposition module in HS-FPN and SFDNet?
  2. Why the specific semantic interaction and detail-refinement designs are necessary?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.