acceptodds
Under review as a conference paper at ICLR 2027

Four Cells Are Not the Same Dose: Size-Conditioned Spatial Lesions in Detection Transformers

Abstract

A spatial lesion that removes four feature cells can erase most of a small object's footprint while barely affecting a large one. We test whether this mismatch changes conclusions drawn from decoder cross-attention in frozen DETR and Conditional DETR checkpoints. On a preregistered 4,000-image COCO holdout, the small-minus-large local-quality contrasts for four in-box versus four out-of-box cells are 0.02136 and 0.02325. When we instead remove the same 0.02 clean mean-head attention mass from each eligible query, both 95% image-bootstrap intervals fall inside the prespecified ±0.005 equivalence region. This low-dose result applies only to eligible GT-matched objects: the four-cell gate excludes 60.3% of candidate small objects. At an exploratory dose of 0.40, size contrasts rise again, so equal mass is not a general explanation of size sensitivity. Prediction-selected detector AP effects are small and differ from the GT-matched local estimand. An exploratory deformable sampling-footprint candidate improves small-object AP in an author-designated summary, but the baseline fails a prespecified feasibility gate and conflicting source records preclude a validated improvement claim. The practical contribution is a dose-aware spatial-ablation protocol that reports eligibility and the detector-level readout alongside query-level effects.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Object Detection, Detection Transformers, Cross-Attention Interpretability, Spatial Ablation, Attention-Mass Matching, Object-Size Sensitivity.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_V6fvOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
3
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This paper investigates the issue of fairness in spatial attention ablation experiments for Detection Transformers. The authors point out that when a fixed number of feature cells are removed, small objects, which inherently occupy a smaller spatial area, may undergo a much stronger actual intervention than large objects. Therefore, a fixed number of cells does not necessarily correspond to the same intervention intensity. To address this issue, the paper compares two settings: removing a fixed number of cells and removing a fixed amount of attention mass. Experiments are conducted using DETR-R50 and Conditional DETR-R50. The results show a significant difference between small and large objects when four cells are removed, while this difference is substantially reduced under the equal-mass setting. The paper further presents exploratory analyses across different layers, checkpoints, and detector architectures, including sparse and deformable detectors.

Critical strengths
  1. The paper highlights an easily overlooked issue in experimental design: that removing a fixed number of feature cells does not necessarily equate to applying an intervention of equal strength to targets of different scales. This observation is sound and holds methodological significance.

  2. The authors provide a comprehensive account of experimental limitations, exploratory analyses, and protocol changes, clearly distinguishing between results that support strong conclusions and those that serve merely as exploratory evidence.

Critical weaknesses
  1. The primary experiments utilize earlier architectures such as the original DETR and Conditional DETR, while the subsequent R101 experiments belong to the same model family; however, as the authors themselves acknowledge that the R101 and R50 experiments employed different decoder layers, this portion can only serve as an exploratory extension rather than constituting a rigorous cross-checkpoint confirmation.

  2. A significant portion of the text is dedicated to detailing protocol changes, exploratory status, audit limitations, failed gates, post-hoc analyses, and conclusions that could not be supported. While such transparency is commendable, the frequency with which these caveats appear makes the article read more like an audit report of the experiment than a research paper centered on a clear scientific question.

Questions for authors

Can the authors re-verify the core conclusions regarding "fixed-count" versus "equal-mass" using a more modern detection transformer, while maintaining intervention layers and evaluation protocols that are entirely consistent with the main experiments?

Reviewer_HtAmOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how spatial attention perturbations affect small and large objects in detection transformers. The authors compare fixed-cell removal with equal-attention-mass removal using DETR and Conditional DETR, and further explore a sampling-footprint modification for small-object detection.

Critical strengths

The paper raises a reasonable methodological concern about comparing spatial perturbations across objects of different sizes.

The experiments include controlled perturbations, statistical analysis, and several sensitivity checks.

Critical weaknesses

The main experiments are conducted on DETR and Conditional DETR, both relatively old architectures. It is unclear whether the conclusions generalize to modern multi-scale and deformable detectors, such as RT-DETRv2 or DEIM. Although the paper includes an exploratory DEIMv2 experiment, it does not successfully validate the main findings on this architecture.

The paper mainly reports query-level quality differences, while the corresponding changes in detector-level AP are very small. It is unclear how these findings translate into meaningful improvements in detection performance or model interpretability.

The FFOS experiment reports positive results, but the baseline achieves only 0.432 APs and fails the authors' predefined feasibility criterion. Therefore, the experiment does not convincingly demonstrate the effectiveness of the proposed modification.

The submission is limited small-object coverage, lack of validation on modern architectures, and weak connection between the measured effects and practical detection performance.

Questions for authors

Refer to critical weaknesses

3 comments
Reviewer_HtAmOpenReview ↗

Final recommendation

reject. concerns remained.

Thank you for the review. You asked whether any of this matters for actual detection performance. It does, and showing that became the centerpiece of our response.

Detector-level AP. The AP check in the submission applied one shared mask per image to every query, so most detections were lesioned away from where they were looking. The new experiment gives each query a lesion inside its own predicted box, then scores the detections with the official COCO evaluator. S is the extra AP lost by small objects compared with large ones:

Lesion (main layer)DETR-R50Conditional DETR-R50
Four cells inside the box-2.47 [-2.82, -1.13]-0.98 [-1.59, -0.30]
Equal mass, m = 0.02-0.08 [-0.20, -0.00]-0.03 [-0.13, +0.07]
Four cells outside the box-0.02 [-0.07, +0.01]-0.00 [-0.02, +0.02]

[--- split here ---]

Deleting four cells costs small objects 2.5 and 1.0 AP points more than large ones. Match the dose and the gap shrinks to 0.08 and 0.03 points; the difference between the two lesions is -2.39 [-2.73, -1.04] and -0.94 [-1.55, -0.27] points. Deleting the same number of cells outside the box changes nothing. If the lesion is applied at every decoder layer, the fixed-count gap grows to 4.7 and 9.9 points.

The reason is easy to see in the dose. Most small boxes cover fewer than four cells, so "four cells" wipes out everything inside them: in DETR-R50 it removes a mean attention mass of 0.34 from small boxes and only 0.03 from large ones. Under the equal-mass lesion every object loses 0.02.

At a high dose (m = 0.40) the two models diverge: DETR-R50 still shows a gap (-1.53), Conditional DETR-R50 does not. This fits the submission's high-dose result, and the revision limits the conclusion to low dose accordingly.

The practical takeaway is simple. In a dense detection transformer, a fixed-count spatial ablation hits small objects much harder. Anyone concluding from such an ablation that small objects "rely more on attention" should first report how much attention was actually removed at each size, or compare at equal dose. The revised paper makes this a concrete recommendation.

Modern detectors. In DEIMv2-S and RT-DETRv2-R50vd, sampling scales with the box, so four sampling points remove about the same attention at every size (0.87x and 0.85x, versus about 17x in DETR), and the size effect does not appear. They also let us test 94-97% of small objects, against 40% in DETR. Full numbers are in our response to Reviewer V6fv.

FFOS. We agree with your assessment. The new dose measurements also show that deformable detectors already treat small and large objects evenly, so FFOS addresses a different question from this paper. We removed it to keep the paper focused (see the general comment).

Coverage. The AP experiment includes every query, with no eligibility filter. And since all of this was run on the reused COCO holdout, we are confirming it on VOC 2007 test under a frozen registration.

AuthorsOpenReview ↗

Final Response

Thank you for the review. This is our final response, and it refers to the revised PDF. The concerns you raised are the ones we spent most of the revision on. They led to new experiments on three modern detectors, a detector-level AP analysis on COCO and two VOC splits, and a substantial rewrite of the text and figures.

Modern detectors. We added DEIMv2-S, RT-DETRv2-R50vd and Deformable DETR (Sec. 3.4 and 3.5). The result is more informative than a straight replication. Their sampling points scale with the reference box, so four points remove about the same share of attention from small and large objects (small/large ratios of 0.87 and 0.85), and the size contrast stays within the margin. If we instead delete a fixed spatial area in the same checkpoints, the dose ratio becomes 52 and 84 on VOC 2007, and the size contrast comes back (+0.052 and +0.029). So it is the budget, not the architecture family, that decides whether a size gap appears. These within-model predictions were registered before VOC 2007 test was used, and all six were confirmed.

Detector-level AP. You were right that the local effects in the original submission did not translate into much AP. The revised AP analysis lesions every query inside its own predicted box. On COCO, up to four cells cost small objects 2.47 AP points more than large ones in DETR-R50 and 0.98 more in Conditional DETR-R50. On VOC 2007 test, the registered estimates are 2.45 and 5.39, and at matched low dose both gaps fall inside ±0.5 AP.

The more practical consequence is in Sec. 3.3. Under a four-cell budget, which of the two detectors looks more size-sensitive depends on how many decoder layers are lesioned: the comparison moves by 6.7 to 9.6 AP points across COCO and both VOC splits. A box-relative budget removes most of that dependence on COCO and on VOC 2012 (6.56 [4.04, 10.05], registered); the VOC 2007 test was inconclusive. For interpretability work, this means a fixed count can change the conclusion of a model comparison.

FFOS. Agreed. With a baseline below our own feasibility criterion, the experiment cannot support an improvement claim. Because FFOS addresses a different question from the lesion study, we removed it, and the paper now focuses on the measurement question alone.

Small-object coverage. The dense four-cell local analysis still covers 39.7% of candidate small objects, which the paper states. The deformable four-point analysis covers 94% to 97.5%, and the AP analysis has no eligibility restriction.

We also rewrote the paper to make the main question and its practical consequence easier to follow, and redrew all figures. Thank you again; we hope the revised version addresses your concerns.

Reviewer_BuMgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether fixed-count spatial lesions are comparable interventions across object sizes in frozen DETR-R50 and Conditional DETR-R50. On a 4,000-image COCO holdout, deleting four in-box cells produces larger small-versus-large local-quality contrasts than deleting an equal number of out-of-box cells. Matching the removed clean mean-head attention mass to 0.02 yields contrasts within the authors’ prespecified equivalence region, but this result is conditional on eligible GT-matched objects and does not hold as a general claim across doses. The four-cell eligibility rule excludes 60.3% of candidate small objects, and exploratory high-dose results show renewed size differences. Prediction-selected AP changes are small and are distinct from the local query-level estimand. The paper’s main contribution is a dose-aware protocol for spatial-attention ablations; its sparse-attention/FFOS extensions do not establish a validated improvement claim.

Critical strengths
  1. The paper identifies a concrete confound in fixed-cell ablations: the same number of deleted cells can remove very different amounts of attention mass across object sizes. Directly comparing fixed-count and mass-matched interventions makes this methodological point testable.
  2. The authors report the eligibility loss and distinguish local query-level outcomes from detector-level AP, while clearly labeling many extensions as exploratory. This improves the interpretability of the main result.
Critical weaknesses
  1. The low-dose equivalence result is conditional on objects meeting the four-cell eligibility gate, which excludes 60.3% of candidate small objects. The ±0.005 margin has no external AP or perceptual validation, and the high-dose results show that equal mass does not generally remove size-conditioned effects. The main conclusion should therefore remain narrowly scoped.
  2. The primary per-image records and full bootstrap arrays are not included in the delivered materials, and a public artifact release is still pending. This makes independent verification difficult. The auxiliary FFOS result also fails its stated feasibility gate; the submission’s abstract notes conflicting source records, so it should not support a validated improvement claim.
Questions for authors
  1. Could you clarify how readers should interpret the ±0.005 equivalence margin in practical terms, given that it has no external AP or perceptual anchor? Would you characterize the result strictly as protocol-defined equivalence at dose 0.02, rather than as evidence that size sensitivity is generally absent?
  2. Can you provide or commit to releasing the primary per-image records, bootstrap draws, and scripts needed to independently reproduce the main tables? Could you also clarify the conflicting FFOS source records mentioned in the abstract?
3 comments
Reviewer_BuMgOpenReview ↗

Final recommendation

reject. concerns remained.

Thank you for the careful reading. Your questions about the margin and about verifiability led directly to two of our main changes.

The ±0.005 margin. Agreed: the result is protocol-defined equivalence at m = 0.02 on eligible anchors, not evidence that size sensitivity is absent in general, and the revised abstract and conclusion say exactly that. The margin now also has some scale. The observed intervals sit far inside it (largest endpoint 0.00029), so the decision does not depend on the choice of 0.005. And the new AP experiment shows the same contrast in detection terms: at the same layer, four cells cost small objects 2.5 and 1.0 AP points more than large ones, while at m = 0.02 the gap is 0.08 and 0.03 points (details in our response to Reviewer HtAm). We do not claim an exact conversion between the two scales; the VOC 2007 registration sets its own AP-scale threshold in advance.

Verifiable materials. Our archive kept the hashes and summary outputs of the original run, but not the per-image files or the bootstrap arrays. Rather than ask you to rely on the archive, we regenerated everything from the frozen, preregistered analysis code. The rerun is fully deterministic (two runs give identical files) and reproduces Table 1:

CheckpointQuantitySubmittedRegenerated
DETR-R50Δ4+0.0214 [0.0174, 0.0255]+0.0215 [0.0175, 0.0257]
DETR-R50Δ.02+0.00015 [0.00001, 0.00029]+0.00015 [0.00001, 0.00028]
Conditional DETR-R50Δ4+0.0232 [0.0197, 0.0271]+0.0235 [0.0199, 0.0273]
Conditional DETR-R50Δ.02-0.00002 [-0.00029, 0.00024]-0.00002 [-0.00030, 0.00024]

Every anchor count matches exactly (839 eligible small anchors; 826 and 836 at m = 0.02), every decision in Table 1 is unchanged, and the estimates agree to the third decimal place; the rerun uses a newer software stack. The anonymous repository, released together with the revised paper, will contain these per-image records, all bootstrap draws, the scripts, environment files and per-file hashes, so anyone can recompute every number above.

[--- split here ---]

FFOS. FFOS was an exploratory extension: a way to widen the sampling footprint of small objects in deformable detectors, motivated by a measurement suggesting they were under-covered. The new dose measurements change that picture. Deformable detectors already give small and large objects nearly the same dose (0.87x and 0.85x, versus about 17x in DETR-style models), so the imbalance this paper studies belongs to dense grids, and FFOS addresses a different question. We removed it so the paper makes one claim and supports it fully.

On the two records you asked about: the earlier archived CSV and the later re-measured summary behind Table 2 could not be reconciled, because the earlier CSV is not in our current archive. With FFOS removed, Table 2 no longer supports anything in the paper. While re-checking our DEIMv2-S code for the new experiments, we also found and fixed a configuration error that had affected the original FFOS premise measurement. Our implementation now reproduces the official model's outputs exactly, and the new sampling-point lesion leaves the output unchanged at zero dose, as it should. The appendix keeps a short record of the FFOS runs.

Eligibility and high dose. The 60% exclusion is a feature of dense grids; in DEIMv2-S and RT-DETRv2, 97.5% and 94.0% of small objects can be tested, and the new AP experiment has no eligibility filter at all. On high dose we agree, and the new data support your reading: at m = 0.40 the AP gap remains in DETR-R50 but not clearly in Conditional DETR-R50. The conclusion is therefore stated for low dose only.

AuthorsOpenReview ↗

Final Response

Thank you for the careful reading. This is our final response, and it refers to the revised PDF. Both of your questions shaped the revision directly. Since the review, we added three deformable detectors, aligned the R101 checkpoints with the R50 layer, ran a registered replication on all of VOC 2007 test plus a registered follow-up on VOC 2012, and rewrote the text and figures. The answers below point to the relevant sections.

On the ±0.005 margin. Yes, we now describe the result exactly that way. The margin is protocol-defined and has no external AP or perceptual calibration. The result at m = 0.02 says that the absolute small-minus-large contrast is negligible at that dose, not that size sensitivity is absent. Sec. 3.2 now also gives the other side: at matched absolute doses from 0.1 to 0.6, small objects lose 2.6 to 3.8 times as much local quality as large ones in both R50 models (Appendix Table 12).

The main claim of that section has changed. When dose is measured relative to each object's in-box attention, it accounts for 68% and 98% of the four-cell gap on COCO, and 61% and 98% on VOC 2007 (registered). Removing an equal fraction of in-box attention leaves a gap 70 to 80% smaller than four cells do. The calibration caveat is in the Limitations paragraph.

On eligibility. The 60.3% exclusion still applies to the dense four-cell local analysis, and we say so in Sec. 3.1 and in the Limitations. Two additions reduce how much rests on that subset:

  • The four-point rule in deformable detectors admits 97.5% (DEIMv2-S) and 94.0% (RT-DETRv2) of candidate small objects.
  • The detector-level AP analysis lesions every query, so it has no eligibility gate. Boxes with fewer than four cells lose all of them, and Table 2 reports how often this happens.

On release. Per-image records, bootstrap draws, registration texts and the scripts that rebuild the tables are at https://anonymous.4open.science/r/size-conditioned-lesions-3C09/. Regenerating the primary records with the frozen code changes the two fixed-count estimates by at most 0.0002 and changes no decision (Appendix Table 5).

On FFOS. We agree it should not support an improvement claim, and the revised paper no longer makes one. FFOS is a training-time modification and answers a different question from the lesion study. Its baseline fell short of our own feasibility criterion. Rather than build a claim on that evidence, we removed FFOS and kept the paper focused on the lesion analysis.

Thanks again for the review. We hope the revised version addresses your concerns.

Comments on the paper

5 comments

Thank you for the review. Your question was the right one to ask, and we ran exactly that test.

R101 at the same layer. You were right that the R101 runs in the submission used a different layer. At the main layer, with everything else as in the main analysis, both checkpoints reproduce the R50 result:

CheckpointΔ4, four cellsΔ.02, equal mass
DETR-R101+0.0226 [0.0183, 0.0273]+0.00003 [-0.00011, +0.00016]
Conditional DETR-R101+0.0220 [0.0186, 0.0258]-0.00004 [-0.00039, +0.00037]

A four-cell lesion hurts small objects clearly more, and at a matched low dose the difference is close to zero and inside the registered margin.

Modern deformable detectors. We then tested DEIMv2-S and RT-DETRv2-R50vd at the same layer and on the same holdout. These models read features through a few sampling points per query instead of a dense grid, so the lesion removes sampling points instead of cells, one query at a time; for a fair comparison the DETR rows below use the same per-query lesion. Everything else, including how dose is measured, is unchanged.

CheckpointMass removed, small / largeΔ4Δ.02
DETR-R5017.3x+0.0253 [0.0206, 0.0305]+0.00007 [-0.00007, +0.00020]
Conditional DETR-R5016.8x+0.0259 [0.0220, 0.0302]-0.00007 [-0.00031, +0.00016]
DEIMv2-S0.87x-0.0005 [-0.0009, -0.0001]+0.00007 [-0.00007, +0.00020]
RT-DETRv2-R50vd0.85x-0.0011 [-0.0022, -0.0001]-0.00015 [-0.00079, +0.00034]

The first column tells the story. In DETR-style models, four cells take about 17 times more attention away from a small object, and small objects suffer much more. In the deformable models, sampling scales with the box, so four points take about the same amount from every object, and the size effect essentially disappears (Δ4 is 20 to 50 times smaller, and if anything slightly negative, matching the slightly smaller dose). In both families the size effect follows the dose. The lesions clearly work in these models: their effect grows steadily as the dose increases. The deformable models also leave far fewer objects out: 94-97% of small objects can be tested, against 40% in DETR.

So the conclusion carries over to modern detectors in the most informative way: it holds wherever the dose is unequal, and it tells practitioners which architectures need dose matching. We are also testing these predictions on VOC 2007 test under a frozen registration (general comment).

On presentation, we fully agree. The revision is rewritten around one question and three findings: a fixed cell count is not a fixed dose in dense detection transformers; this confounds size comparisons, visibly so in AP; and deformable detectors, where the dose is balanced, do not show the effect. Protocol history moves to the appendix, and FFOS is removed so the paper stays focused on this claim.

AuthorsOpenReview ↗

We thank the AC and all reviewers for their time and feedback. Below is our general comment addressing the reviewers’ concerns.

Thank you for the review. Your question was the right one to ask, and we ran exactly that test.

R101 at the same layer. You were right that the R101 runs in the submission used a different layer. At the main layer, with everything else as in the main analysis, both checkpoints reproduce the R50 result:

CheckpointΔ4, four cellsΔ.02, equal mass
DETR-R101+0.0226 [0.0183, 0.0273]+0.00003 [-0.00011, +0.00016]
Conditional DETR-R101+0.0220 [0.0186, 0.0258]-0.00004 [-0.00039, +0.00037]

A four-cell lesion hurts small objects clearly more, and at a matched low dose the difference is close to zero and inside the registered margin.

Modern deformable detectors. We then tested DEIMv2-S and RT-DETRv2-R50vd at the same layer and on the same holdout. These models read features through a few sampling points per query instead of a dense grid, so the lesion removes sampling points instead of cells, one query at a time; for a fair comparison the DETR rows below use the same per-query lesion. Everything else, including how dose is measured, is unchanged.

CheckpointMass removed, small / largeΔ4Δ.02
DETR-R5017.3x+0.0253 [0.0206, 0.0305]+0.00007 [-0.00007, +0.00020]
Conditional DETR-R5016.8x+0.0259 [0.0220, 0.0302]-0.00007 [-0.00031, +0.00016]
DEIMv2-S0.87x-0.0005 [-0.0009, -0.0001]+0.00007 [-0.00007, +0.00020]
RT-DETRv2-R50vd0.85x-0.0011 [-0.0022, -0.0001]-0.00015 [-0.00079, +0.00034]

The first column tells the story. In DETR-style models, four cells take about 17 times more attention away from a small object, and small objects suffer much more. In the deformable models, sampling scales with the box, so four points take about the same amount from every object, and the size effect essentially disappears (Δ4 is 20 to 50 times smaller, and if anything slightly negative, matching the slightly smaller dose). In both families the size effect follows the dose. The lesions clearly work in these models: their effect grows steadily as the dose increases. The deformable models also leave far fewer objects out: 94-97% of small objects can be tested, against 40% in DETR.

So the conclusion carries over to modern detectors in the most informative way: it holds wherever the dose is unequal, and it tells practitioners which architectures need dose matching. We are also testing these predictions on VOC 2007 test under a frozen registration (general comment).

On presentation, we fully agree. The revision is rewritten around one question and three findings: a fixed cell count is not a fixed dose in dense detection transformers; this confounds size comparisons, visibly so in AP; and deformable detectors, where the dose is balanced, do not show the effect. Protocol history moves to the appendix, and FFOS is removed so the paper stays focused on this claim.

Reviewer_V6fvOpenReview ↗

Response to the Program Chairs

To provide greater clarity about my review process, I have included my original review comments below for reference:

1, 这测的都是一些老框架,都是2020年的模型,方法也不新鲜,感觉效果好 但本质上更像是一个trick,有那么多后续的新框架,新模型 尝试一下可能更有说服力一些;

2,没看出来有什么insight,只是展示了结果,分析也不太够,甚至都没写完8页;

3,很多新模型 就算测了,其实模型架构变化也比较大,不一定能对齐setting,但是有总比没有好,甚至这些有待优化的问题在放新模型上都不一定算是个问题;

To avoid having my review flagged as AI-generated, I revised the wording and then translated it using Google Translate. However, it was still flagged as AI-generated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.