- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Diagnose-Before-Design, a workflow for analyzing and simplifying the interface between plain foundation Vision Transformers and multiscale object detectors. The approach uses inference-time branch zeroing, spatial-memory masking, energy-matched controls, and cross-image permutation to measure functional dependencies in a trained detector. Experiments on DINOv3-S, DINOv2-S/14, and DEIMv2 show that deep semantic features are important, while their dominant spatial routes depend on the backbone and interface topology. Factorial experiments further demonstrate interactions between depth treatment and spatial topology. Guided by these observations, the authors develop redesigns that approximately preserve Full-COCO AP while reducing parameters.
Critical strengths
-
Useful diagnostic perspective. The paper measures functional reliance on interface paths rather than relying solely on representation similarity or final architecture-level AP.
-
Complementary intervention controls. Branch zeroing, memory masking, energy matching, and permutation provide converging evidence for image-specific deep-feature dependence.
-
Insightful depth–topology interaction. The DINOv3 factorial reports interactions of +1.864 Raw AP and +1.395 EMA AP. DEIMv2 provides additional evidence, with a mean interaction of +1.254 Raw AP.
-
Appropriate qualification of the findings. The authors acknowledge that dependence is checkpoint-specific and that the observed overall accuracy differences should not be presented as established improvements.
Critical weaknesses
-
The diagnosis-to-design link is only partially demonstrated. The central DINOv3 redesign uses a pre-existing deep-heavy weighting vector whose exact derivation is not recoverable.
-
Limited comparison with simpler design strategies. The paper does not establish how much diagnostic guidance improves candidate selection over straightforward alternatives such as uniform weighting, deepest-feature-only inputs, or a comparably budgeted heuristic search.
-
Search-efficiency benefits are not quantified end to end. Linear scaling in inference evaluations is attractive, but total diagnostic cost and training savings relative to a concrete train-and-compare budget are not reported.
-
The endpoint benefits include trade-offs. DINOv3 reduces GFLOPs by 3.65%, and DINOv2 by 1.44%. Small-object AP decreases in both redesigns. These results support capacity reduction with near-parity aggregate accuracy, with less evidence for uniformly improved detection performance.
Questions for authors
-
Can you provide a reproducible rule that converts intervention scores into feature weights or architecture choices? How does it perform relative to the inherited deep-heavy vector?
-
How sensitive is the DINOv3 interaction to alternative deep-heavy coefficients, including simpler hand-specified schedules?
-
Does the complete DINOv3 four-cell interaction remain positive across additional training seeds?
-
Under an equal total compute budget, how does diagnosis-guided selection compare with simple heuristic or random candidate selection? Please include the cost of all diagnostic evaluations.
-
How stable are the dependency rankings across training seeds and training stages?
-
Can you explain the small-object AP reductions?