- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes MoMI-RC for multi-modal object re-identification using RGB, near-infrared, and thermal images. It combines identity-supervised global modality weights with sample-specific adjustments, and preserves pairwise and three-way modality interactions in separate feature blocks. Modality-subset training and consistency regularization further support robust representation learning. Experiments on three person and vehicle benchmarks report competitive retrieval performance, with component ablations conducted on RGBNT201.
Critical strengths
The proposed reliability calibration and multi-order interaction modules achieve strong retrieval performance across three benchmarks, demonstrating the practical effectiveness of the overall framework for both person and vehicle re-identification.
Critical weaknesses
-
The introduction does not clearly explain the concrete challenges in existing methods or why the proposed RC and MoMI designs address them. The reliance on abstract terminology makes the motivation and intended contributions difficult to assess.
-
Beyond improvements in Rank-1 and mAP, the main text provides limited direct evidence supporting the core mechanisms. In particular, it remains unclear whether the global anchors reliably capture modality reliability and whether the separate interaction blocks preserve complementary information. This limits confidence in the explanation for the reported gains.
Questions for authors
-
The introduction does not clearly explain the concrete limitations of existing methods or how RC and MoMI address them. Could you illustrate these limitations with a concrete example? A schematic comparing existing fusion approaches with the proposed design would help clarify the motivation.
-
The component ablations are limited to RGBNT201. Could you report the main component ablations on RGBNT100 and MSVR310 as well, to establish whether the benefits of RC, MoMI, subset training, and consistency regularization hold across both person and vehicle ReID?
-
The main text does not clearly demonstrate whether the proposed mechanisms work as motivated except rank and mAP performance. But I want to have a look more direct evidence supporting the core designations. Could you display the other evidence?
-
Could you distinguish modalities within each identity in the t-SNE visualization, for example using colors for identities and marker shapes for modalities? If the plotted descriptor is a fused tri-modal representation, a separate visualization of modality-specific features would be more informative.
I will improve the rating and confidence if the rebuttal provides clearer explanations and convincing supporting results.
3 comments
Thanks for your response. The response has not fully addressed my concerns. So I will not improve rating.
Follow-up on the remaining concerns
Thank you for reading our response. Since the discussion period is nearly over, we would appreciate knowing which concern remains open, so that we can answer it specifically. For the AC's convenience, the evidence for each of your weaknesses and questions is summarized below; [W1]–[W4] in our first response correspond to Q1–Q4.
- Weakness 1 / Q1 (concrete limitations and a schematic). Figure 1 of the revised PDF illustrates the two limitations we address with concrete examples: overall modality reliability and per-image degradation are not modeled separately, and cross-modal interactions are not organized by source. Table 3 isolates the second point: with the same products, separate blocks give 84.2% mAP and a merged block gives 83.1%.
- Q2 (ablations on RGBNT100 and MSVR310). The component ablations on RGBNT100 (85.4 → 89.6) and MSVR310 (65.8 → 70.5) show the same trend as RGBNT201, with mean ± std over 3 runs.
- Weakness 2 / Q3 (direct evidence for the mechanisms). Global anchors: the learned anchors (TIR 1.178 > RGB 1.026 > NIR 0.827) follow the single-modality test ranking, although they are learned only from training-set identity distances (Fig. 4); under 75% occlusion, RC lowers the weight of the occluded modality for 97.7–99.9% of the 836 queries (Fig. 9). Separate interaction blocks: with a missing modality, the remaining pairwise block keeps the full model at 79.4% mAP for R+T input (Table 7), and separate blocks outperform merged ones (Table 3). Descriptor level: the positive–negative cosine gap rises from 0.250 to 0.403 (Fig. 5).
- Q4 (modality-wise t-SNE). Figure 10 in Appendix A.7 visualizes RGB, NIR, and TIR features separately, with colors for identities, and shows more compact clusters for MoMI-RC in each modality.
If any of these points does not match what you had in mind, a brief note would let us respond before the deadline.
Clarified motivation and additional supporting evidence
Thank you for the constructive comments. We address each point below.
[W1] Concrete limitations and motivation. RC and MoMI address the two limitations below, and Figure 1 of the revised PDF illustrates both.
Limitation 1: overall modality reliability and per-image degradation are not modeled separately. A modality's weight depends on two different factors. The first is whether the current image is degraded; e.g., an occluded RGB image should lower the RGB weight for that image only. The second is how well the modality distinguishes identities overall; e.g., NIR has the lowest single-modality mAP on RGBNT201 (28.2%, Table 7). Input-adaptive methods such as DeMo predict weights from the current input, but have no identity-supervised parameter for the second factor. RC models the two factors separately. A global anchor, supervised by identity distances in training batches, encodes the overall reliability of each modality, and the learned NIR anchor is the lowest (Fig. 4). A bounded sample residual then adjusts the weight for the current image; for example, occluding 75% of an RGB image lowers its weight from 1.42 to 0.94 (Fig. 9).
Limitation 2: cross-modal interactions are not organized by source. Direct concatenation, common in multi-modal ReID, keeps only first-order features. If the interactions are summed or averaged into one vector, it may no longer be clear which modality pair the evidence comes from. For example, if sample A has a strong RGB–NIR and a weak RGB–TIR interaction and sample B the reverse, averaging makes them identical. MoMI places each of the RN, RT, NT, and RNT interactions in its own compact block and concatenates them with the first-order features. In Table 3, with the same products, separate blocks give 84.2% mAP and a single merged block gives 83.1%. Under the same setting, MoMI also outperforms our bilinear-, TFN-, and LMF-style heads (82.4–82.9%) with fewer head parameters (see our response to Reviewer M9zo).
[W2] Ablations on RGBNT100 and MSVR310. Following Table 2, mAP (mean ± std over 3 runs):
Both datasets follow the RGBNT201 trend: MoMI and RC each improve over the baseline, their combination outperforms either alone, and subset training and consistency regularization raise mAP further. The full model's mean mAP is 4.2 and 4.7 points above the baseline, so the components give consistent gains for both person and vehicle ReID.
[W3] Direct evidence beyond Rank and mAP.
Global anchors reflect modality reliability. The anchors learned on RGBNT201 are 1.178 for TIR, 1.026 for RGB, and 0.827 for NIR (Fig. 4). They are learned only from training-set identity distances, yet follow the same order as single-modality test mAP: TIR 56.1%, RGB 44.5%, NIR 28.2% (Table 7).
Sample weights respond to degradation. Following Fig. 9, we occlude the central 75% of one modality for each of the 836 RGBNT201 queries, once per modality (2,508 pairs in total):
Separate interaction blocks. Structurally, each block corresponds to one modality combination. When a modality is missing, only the blocks involving it are zeroed, and the pairwise block of the other two modalities remains usable; with R+T input, for example, the full model reaches 79.4% mAP (Table 7). Empirically, with the same products, separate blocks outperform merging (Table 3: 84.2 vs. 83.1), and keeping both second- and third-order interactions gives the best mAP (Table 8).
Descriptor-level separation. On the same positive and negative query–gallery pairs, the mean cosine-similarity gap rises from 0.250 for the baseline to 0.332 with MoMI and RC, and to 0.403 for the full model (Fig. 5).
[W4] Modality-wise t-SNE. Each point in Fig. 3 is a fused tri-modal descriptor, so modalities cannot be separated there. We therefore visualize the RGB, NIR, and TIR features separately, with color denoting identity; for each modality, the baseline and MoMI-RC panels use the same samples. For the displayed identities, MoMI-RC gives more compact clusters than the baseline in every modality, which qualitatively indicates that the single-modality features also improve. This figure is included as Figure 10 in Appendix A.7 of the revised PDF.