acceptodds
Under review as a conference paper at ICLR 2027

Small Objects Inside the Grid: Activating Latent Scale Knowledge in Frozen Detectors via Training-Free Grid Reparameterization

Abstract

Small object detection is central to high-resolution visual settings such as aerial perception and long-range surveillance. Its core challenge is not limited to scarce target pixels. It also concerns how continuous changes in input scale preserve a consistent semantic-geometric correspondence on the discrete multi-scale grids of a frozen Transformer detector. This internal scale-adaptation mechanism remains poorly understood. Existing methods commonly rely on retraining, architectural expansion, or black-box multi-scale fusion. These approaches implicitly assume that small-object capability must be reacquired through external supervision or that additional resolution can be directly exploited by the model. We challenge this assumption and posit that a pretrained detector already encodes scale-related positional knowledge, while fixed positional embeddings, decoder anchors, and validity masks constrain its use when the input shape changes. To test and exploit this mechanism, we propose Adaptive Causal Grid Intervention (ACGI), a training-free framework that synchronously reconstructs internal spatial tensors from actual feature shapes. ACGI characterizes the response discrepancy between two internal coordinate systems as cross-grid geometric consistency and uses annotation-free spatial-compression routing to select an intervention scale. It requires no gradients, target-domain annotations, or parameter updates. Experiments on Microsoft Common Objects in Context (COCO), TinyPerson, and VisDrone2019 with multiple frozen DEtection TRansformer (DETR) baselines show consistent improvements in overall and small-object detection across different training strategies and decoder sampling mechanisms. Reliability ranking, scale diagnosis, and deletion analyses further demonstrate that cross-grid consistency faithfully captures the dependence of predictions on internal spatial representations. These results recast scale adaptation in small object detection as the identification and activation of latent spatial knowledge in frozen models, rather than the simple addition of input pixels or new downstream parameters.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Small Object Detection, Detection Transformers, Training-Free Adaptation, Mechanistic Interpretability, Counterfactual Intervention, Query Trajectories

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Z9S5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This submission tackles small object detection by proposing a post-processing framework designed to improve performance for tiny objects without requiring detector retraining. The method consists of three main components:

  1. Positional parameter recalculation: Resizing the input image and forwarding it to the encoder to obtain multi-scale feature maps that differ from the training size. The authors compute new positional embeddings and decoding anchors (as outlined in Section 3.2).
  2. Multi-scale result merging: Performing object detection on both the original and resized images, then merging the outputs using 2D IoU. This strategy combines predictions with high cross-scale overlap while integrating newly discovered tiny objects from the resized image.
  3. Heuristic scaling factor selection: Dynamically choosing scaling factors and merging strategies based on the input image area relative to the training size.

While the quantitative results show that the approach can improve tiny object detection, the paper suffers from several major weaknesses, including a lack of baseline comparisons, unanalyzed computational overhead, and limited conceptual novelty. Consequently, the manuscript is not yet ready for publication at ICLR and requires substantial improvements.

Critical strengths

The ablation studies are extensive and well-designed, effectively validating the impact of different parameter choices within the proposed framework.

Critical weaknesses
  1. Lack of comparison with existing post-processing approaches: The experiments only demonstrate that multi-scale inference and box merging boost performance. However, this outcome is expected since enlarging input resolutions naturally aids small object detection. The paper fails to compare its method against other established post-processing techniques (such as image slicing/tiling strategies). To demonstrate a rigorous technical contribution, the authors should use the same base detector and compare their framework against alternative post-processing baselines.
  2. Missing computational overhead analysis: Because this approach requires running object detection at multiple input scales during inference, it inevitably increases latency and memory overhead compared to a single-pass baseline. The paper lacks any concrete profiling or analysis of this computational cost. Other weaknesses and issues are in the following "questions for authors" section.
Questions for authors

Limited novelty: The idea of multi-scale prediction and output merging is a common practice in object detection, which limits the novelty of the work.

Dependence on manual hyper-parameter tuning: Although the method is advertised as "training-free," it relies on hand-tuned heuristics and hyper-parameters to select scaling factors for images of varying sizes. Tuning these parameters requires manual validation, making the approach vulnerable to performance drops on unseen datasets with different data distributions.

Other issues:

  1. There is a discrepancy where the abstract on the OpenReview page does not match the one in the submitted PDF.
  2. The introduction and related work sections suffer from low readability. The language is overly abstract and relies on terminologies that are uncommon in the object detection community (e.g., "Controlled Scale-Coordinate Counterfactual", "Mechanism-Oriented Paradigm"), which obscures the core ideas rather than clarifying them.
Reviewer_1uoGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ACGI is presented as a training-free "causal intervention" on a frozen detector's coordinate system. Positional embeddings, anchors, and masks are regenerated for a larger input, and the agreement between native-grid and enlarged-grid predictions is presented both as an explanatory signal and as the basis for fusion. The paper's central claim is that pretrained detectors hold "latent scale knowledge" that is suppressed by preset spatial tensors and can be "activated" by grid alignment. Evidence is given through detection gains and a five-stage mechanistic analysis (diagnosis, intervention, internal response, positive control, deletion).

Critical strengths
  1. The question is worth asking. How a frozen detector's positional machinery behaves under input-scale changes is a reasonable scientific question, and small-object detection is a natural testbed.
  2. Cross-grid agreement is a cheap, annotation-free reliability cue. Its AUCs of 0.72 / 0.81 against GT overlap suggest it could be useful for selective inference.
Critical weaknesses
  1. The causal and mechanistic claims are not supported by the experimental design. The "controlled counterfactual" changes the input resolution and the coordinate tensors together, so it cannot attribute any effect to tensor alignment as such. The control that would isolate it is missing: the same high-resolution input with (a) regenerated tensors vs. (b) interpolated or stale tensors. The "latent scale knowledge" has a simpler explanation: RT-DETR-family models are trained with multi-scale resizing (about 480–800 px), so they already work at 800 px. That is training-time scale augmentation, not knowledge that needs activating. The deletion study shows that removing candidates both branches agree on hurts AP. That is expected, since TTA agreement is a known correctness signal, and it shows correlation with true positives rather than "causal importance" of an internal positional mechanism. The positive-control gaps are a few tenths of AP, with only two random runs.
  2. Novelty over multi-scale testing and box fusion is thin, and the framing overstates it. The related-work section calls TTA methods "black-box" and "output-level", but ACGI's own fusion is output-level class-aware NMS. Its routing is a threshold on image size. Relevant work on resolution generalization is not discussed: positional-embedding interpolation in ViT/DETR, FlexiViT, NaViT, and multi-resolution evaluation of DETRs. Terms such as "Adaptive Causal Grid Intervention", "grid pressure", and "explanatory consensus" describe standard operations and make it harder to see what is actually new.
Questions for authors

Clarifications:

  1. Can you add the isolating control: fixed 800-px input, comparing regenerated tensors against tensors built for the 640 grid and resampled? Also fixed 640-px input with tensors regenerated for a mismatched grid?
  2. As a predictor of GT overlap, how does cross-grid consistency compare with detection confidence alone and with standard TTA agreement (e.g., flip/scale agreement)? If confidence gives similar AUC, the "mechanistic" signal adds nothing.
  3. For each detector, what input scales were used in training? Do the gains persist for a detector trained at a single fixed scale? That would be the real test of "latent" knowledge.

Suggestions: tone down the causal language unless the isolating control supports it.

Reviewer_MFvCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Causal Grid Intervention (ACGI), a training-free method for improving small-object detection with frozen DETR-based detectors. The method regenerates positional embeddings, decoder anchors, and validity masks according to the actual feature grid, and combines predictions across scales through geometric consistency and class-aware fusion. Experiments on COCO, TinyPerson, and VisDrone2019 demonstrate consistent improvements across five pretrained detectors, particularly for small objects.

Critical strengths
  1. Training-free adaptation. ACGI improves small-object detection without modifying model parameters or requiring additional training data.
  2. Consistent improvements across detectors. The method achieves positive results across five DETR variants and three datasets, suggesting good applicability to different detection architectures.
Critical weaknesses
  1. Limited novelty and unclear performance attribution. The method mainly combines dynamic positional encoding, multi-scale inference, and prediction fusion. Table 4 shows that dynamic-grid inference alone achieves nearly the same performance as the complete method (54.14 vs. 54.17 AP), raising questions about the contribution of the proposed cross-grid consistency mechanism.
  2. Insufficient comparison with standard inference strategies. The improvements may largely come from increasing input resolution. Comparisons with conventional multi-scale testing and image slicing under matched computational budgets are needed to establish the advantage of the proposed approach.
  3. Poor figure and table presentation. Several figures and tables are poorly formatted, with overcrowded layouts and small text, making them difficult to read and interpret.
Questions for authors
  1. How does ACGI compare with standard high-resolution inference and multi-scale testing when using the same input resolutions and computational budget?
  2. Table 4 shows that the complete method improves AP by only 0.03 over dynamic-grid inference alone. What is the practical contribution of cross-grid consistency beyond standard prediction fusion?
  3. What is the inference overhead of ACGI compared with the original detectors? Does the improvement justify the additional computational cost?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.