- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies two coupled design choices in vision-language object detectors: where visual and textual representations first interact, and how many times cross-modal interaction is repeated. The proposed Dual-Core architecture constructs learned text cores using a two-layer resampler and visual cores using top-ranked Grounding DINO proposals followed by reference-aware refinement. The main empirical finding is that one post-core interaction in the first decoder layer performs better than a pre-core bridge, a middle-layer interaction, a last-layer interaction, or four interactions spread over the decoder.
Critical strengths
The interventions of when and how often to let vision and language interact, is carefully experimented with. The finding that one interaction can suffice, is interesting. Keeping dense visual memory for deformable attention and retaining the original text tokens at the final matching head looks technically reasonable.
Critical weaknesses
The method figure seems oversimplified and doesn't really help in understanding the method better. The figure has limited value in this way. I want to understand the mechanisms better and the figure doesn't show that.
That one interaction between vision-text can suffice, is interesting, but also a bit disappointing. It is basically what several models do already. The question is whether the finding of retaining text cross-attention only in decoder layer 1, and that this works better than four alternatives, is interesting enough for the ICLR audience.
Only one detector is tested: Grounding DINO architecture. This is insufficient to establish transfer of the claimed principle to other architectures.
The paper does not discuss sufficiently the prior work on gating, sharing or sparsifying cross-modal attention across transformer layers. E.g. query bottlenecks and Q-Former are not considered.
Questions for authors
Why is {1,2,4,6} the representative repeated schedule?
What exactly is new in the “visual core” relative to Grounding DINO’s existing proposal/query?
Can you demonstrate the proposed mechanism directly, for example by measuring how localization, class evidence, or representation similarity evolves after each decoder interaction?
Can you expand related work on cross-modal interactions, multimodal bottlenecks, in order to clearly state the closest prior architecture and the difference.