acceptodds
Withdrawn from ICLR 2027

ART: From Common Semantics to Unique Details via Decoupled Feature Interaction for RGBT Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

RGB–thermal (RGBT) object detection is essential for around-the-clock perception. However, many existing detectors implicitly couple fusion with interaction—pushing aggressive cross-modal mixing early—which can prematurely collapse modality diversity and suppress complementary cues. We argue that effective interaction should be built on the right factors: shared semantics should be aligned, while modality-specific details should be preserved. Based on this insight, we present ART, a decoupled feature interaction framework that follows Decouple Align Interact. ART first disentangles features into cross-modal common semantics and modality-specific unique details using a modality-agnostic shared projection head and modality-dependent heads. It then stabilizes the shared semantic subspace with a correlation-based alignment objective that encourages an identity-like cross-modal channel correlation—strengthening matched-channel semantic consistency while reducing off-diagonal leakage and redundancy. Finally, ART performs cross-modal interaction separately in the common and unique spaces, enabling semantic agreement without erasing fine-grained complementary information. Experiments on public RGBT benchmarks demonstrate that ART achieves state-of-the-art performance.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Object Detection, RGBT, Cross-modal

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_NzmQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

To address the problem that indiscriminate consistency regularization in existing RGB-T fusion may suppress fine-grained complementary cues, introduce inter-modal interference, and reduce the discriminability of fused representations, the authors propose ART, a decoupling-alignment-interaction framework for RGB-T object detection. ART decomposes RGB and thermal features into a shared common semantic space and two modality-specific unique spaces, models them separately, and performs interaction afterward.

Critical strengths

The experiments are relatively complete, and the method achieves SOTA on four datasets.

Critical weaknesses

Beyond Duality [XX], RDCRNet [XX]. “The optimization procedure is provided in Algorithm ??” “Algorithm ??” “The implementation details of the interaction block are provided in Algorithm ??” “As shown in Tab. ??, ART remains stable under mild and moderate training-time misregistration.” “As shown in Fig. ??, the red region at the bottom of the figure is not due to dispersed attention.” Where are the references here?

There is a problem with the ablation experiments. There are three components, but results are provided for only four combinations, so the contribution of each component remains unclear.

There are too few methods in the efficiency comparison. These columns should be added for all four datasets so that readers can intuitively compare different methods.

The compared methods are somewhat old, and there is no comparison with strong baselines from the three major CV conferences or the three major ML conferences in 2025 and 2026. Setting aside the 2026 methods for now, the paper should at least compare with 2025 baselines, such as M²D-LIF and WaveMamba. These two methods have only tens of millions of parameters and FLOPs of only slightly over one hundred G, while their accuracy is close to or even higher than that of your method. Your method has more than six hundred million parameters and more than one thousand GFLOPs. Current methods already achieve dozens of FPS.

[1] Rethinking Multi-modal Object Detection from the Perspective of Mono-Modality Feature Learning, ICCV, 2025.

[2] WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection, ICCV, 2025.

Questions for authors

Everything has already been stated in the weaknesses.

Reviewer_GjvYOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces ART, a Decouple → Align → Interact framework for RGB-thermal object detection. It projects RGB and thermal features into a shared common space and modality-specific unique spaces, aligns the common features through a channel-correlation loss, and interacts with the two spaces separately before detection.

Critical strengths
  1. The paper includes a range of ablation studies and reports parameter counts, FLOPs, and FPS, making its performance gains and computational costs clear.
Critical weaknesses

1.The input modality for the SeaDronesSee experiment is unclear. The paper presents SeaDronesSee as an RGBT benchmark, which however, only contains RGB images. The author should explain where the thermal input required by ART comes from. 2.There are many placeholders remaining in the paper, such as: line 50 Beyond Duality [XX], line 278 in Algorithm ??, line 323 in Algorithm ??, line 744 As shown in Tab. ??. 3. Several recently proposed methods have been missed in the comparison. Such as Efficient RGB-T Object Detection via Sparse Cross-Modality Fusion. 4. The performance gains compared with recent methods are quite marginal. This weakens the contributions claimed by the authors.

Questions for authors
  1. Does the proposed method outperform other methods in Table 2 in terms of speed and parameter count?
2 comments

Concern 1: 1.The input modality for the SeaDronesSee experiment is unclear. The paper presents SeaDronesSee as an RGBT benchmark, which however, only contains RGB images. The author should explain where the thermal input required by ART comes from.

We acknowledge that there are typos in the text, e.g., [XXX]; however, regarding this point, we believe it is a factual error, and we would also like to ask the reviewer why you think there are only RGB images? As far as we know, the dataset we used is an RGB-T dataset. Did you make this judgment based on the output of an AI tool?

I would like to clarify the factual basis of my concern regarding SeaDronesSee. The official documentation describes the standard object-detection benchmarks as RGB datasets.

My earlier statement that SeaDronesSee “only contains RGB images” may have been too broad. Nevertheless, my question about the source of the thermal input was motivated by the official dataset description. My intention was simply to ask the authors to clarify the experimental setup. I apologize for any unintended offense caused by my wording.

Reviewer_GVuYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ART, a Decouple-Align-Interact framework for RGB-thermal object detection. ART uses a shared projection head and modality-specific projection heads to construct common and unique feature branches, applies an identity-like cross-modal channel-correlation loss only to the common branch, and performs interaction separately in the common and unique spaces before detection. The paper reports improvements on FLIR, M3FD, LLVIP, and SeaDronesSee, together with analyses of random-seed stability, misregistration, backbone scale, and computational cost. The overall framework is intuitive, and the reported results demonstrate its potential for multispectral object detection.

Critical strengths
  1. The paper addresses an important problem in RGBT detection. Selectively aligning a shared branch while leaving modality-specific branches unconstrained is intuitive, and the Decouple-Align-Interact organization makes the intended information flow easy to understand.
  2. The paper reports consistent gains on FLIR, M3FD, and LLVIP and includes useful diagnostics beyond a single main table, including ten-seed stability, robustness to training-time shifts, backbone-scale experiments, an alignment-weight sweep, and an explicit efficiency comparison.
Critical weaknesses

The experimental comparison is not sufficiently up to date. Most baselines in the main tables were published before 2026, while recent 2026 RGBT detection methods discussed in the paper are not included in the quantitative comparison. The authors should add recent state-of-the-art methods under a consistent evaluation protocol, or clearly explain why a direct comparison is unavailable. Without these results, the claimed state-of-the-art performance is not fully supported. 2. The manuscript contains numerous unresolved citations and formatting errors, including [XX], Table ??, Algorithm ??, and Fig. ??. There are also duplicated sentences and inconsistent section and appendix references. These issues interrupt the presentation and make it difficult to locate important methodological and experimental details. All citations, cross-references, table and figure numbers, and section links should be carefully checked and corrected. 3. Several parts of the methodology are written in an informal or placeholder-like style. Examples include standalone phrases such as 'The detail of subspace', 'Theory', and 'Proof and explanation', as well as grammatically incomplete descriptions and repeated statements. The methodology section should be rewritten in a precise academic style, with each design choice introduced clearly and the relationship between the equations, modules, and algorithm stated explicitly. 4. The common-unique decomposition lacks an explicit constraint that verifies or enforces the intended separation. The use of a shared common projector and modality-specific unique projectors does not by itself guarantee that common features contain shared semantics or that unique features preserve modality-specific details. As a result, the interpretability of the decomposition remains limited. The authors should provide quantitative or visual evidence that the two types of features capture distinct information, and clarify whether additional supervision or separation constraints are necessary.

Questions for authors

See the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.