- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies weakly supervised visible-infrared person re-identification (WS-VIReID), where identity labels are available within each modality but cross-modal identity correspondences are unavailable during training. To address this issue, the authors propose ReCUE, a correspondence-representation co-evolution framework consisting of three components. First, Global Correspondence Hypothesis Inference (GCHI) estimates correspondence reliability by considering the agreement among Top-K globally feasible one-to-one assignments. Second, Structural Uncertainty-Aware Learning (SUAL) introduces differentiated supervision strategies, including hard supervision, soft supervision, and deferred supervision, according to correspondence reliability. Third, Correspondence-Representation Co-Evolution (CRCE) periodically reconstructs cross-modal correspondences, allowing previously estimated matches to be revised as the representation improves. Experiments on SYSU-MM01 and LLCM demonstrate competitive retrieval performance.
Critical strengths
-
The paper provides useful insights into incorporating structured correspondence uncertainty into cross-modal representation learning. In particular, the combination of global assignment consensus and revisable supervision offers an interesting perspective on mitigating error reinforcement in weakly supervised learning.
-
The experimental results demonstrate competitive performance across multiple benchmarks. These experiments strengthen the empirical evidence supporting the proposed framework.
Critical weaknesses
-
The proposed framework builds upon several established ideas, including global assignment, Top-K matching hypotheses, uncertainty-aware supervision, and alternating optimization. Although their integration is meaningful, the paper could further clarify the distinctive contribution of the proposed combination.
-
A more explicit discussion of how structured assignment uncertainty provides information beyond conventional confidence estimation would help strengthen the novelty claims.
Questions for authors
-
Could the authors provide some additional intuition on why consensus among globally feasible assignments is particularly beneficial for cross-modal supervision compared with conventional local confidence estimation?
-
The correspondence reconstruction strategy appears effective in correcting early matching errors. Could the authors further elaborate on how SUAL and CRCE complement each other at different stages of training?
-
Could the authors briefly report the practical computational overhead of GCHI on SYSU-MM01 and LLCM, such as the approximate time required for each correspondence refresh? This would help readers better understand the efficiency of the proposed framework.
-
The paper already includes extensive experimental comparisons. Could the authors additionally comment on whether the reported improvements remain stable across different independent training seeds?
5 comments
Response to Reviewer kFYm (2/5): Global Consensus versus Local Confidence
Weakness 2: A more explicit discussion of how structured assignment uncertainty provides information beyond conventional confidence estimation would help strengthen the novelty claims.
Question 1: Could the authors provide some additional intuition on why consensus among globally feasible assignments is particularly beneficial for cross-modal supervision compared with conventional local confidence estimation?
A2: Global consensus measures how consistently a match survives competition for exclusive partners. A pair can have high local similarity while another identity competes for the same partner, and feasible assignments resolve these conflicts jointly. TokenMatcher [4] aggregates similarities across token embedding spaces, while ReCUE measures agreement among discrete feasible assignments in Equation (5), Section 3.3 (page 4). Proposition 3 of Appendix A.1 (page 15) constructs two problems with the same selected-edge cost and unique best assignment but different support among their two best assignments, demonstrating information beyond the local score.
Table 12 (page 26) compares reliability scores on the same 395 initial selected edges, treating incorrect matches as the positive class. Uniform Top-K achieves AUROC/AUPRC of 0.803/0.520 versus 0.688/0.407 for local cost, and reliable-set precision of 91.7% versus 84.2% at the same 67.3% coverage. Table 13 (page 26) compares precision across coverage levels and shows uniform Top-K leading at the default, 80%, and 90% coverage. This connects its error-detection advantage to the coverage range used for reliable supervision. Thus, structured consensus exposes global partner competition and provides stronger error detection and cleaner reliable supervision than local confidence at the operating coverage used by ReCUE.
[4] Wang et al. “TokenMatcher: Diverse Tokens Matching for Unsupervised Visible-Infrared Person Re-Identification.” AAAI, 2025.
Response to Reviewer kFYm (3/5): Complementary Roles of SUAL and CRCE
Question 2: The correspondence reconstruction strategy appears effective in correcting early matching errors. Could the authors further elaborate on how SUAL and CRCE complement each other at different stages of training?
A3: Structural Uncertainty-Aware Learning determines supervision within each stage, and Correspondence-Representation Co-Evolution revises the correspondence state between stages. Equation (7) in Section 3.3 (page 5) assigns hard targets to reliable matches, soft targets to ambiguous matches, and intra-modal learning with deferred correspondence supervision to unreliable matches. Equation (9) in Section 3.4 (page 6) rebuilds prototypes, costs, hypotheses, support, and routing after representation updates, allowing matches to receive stronger supervision or be downgraded or replaced. Table H separates corrections from reversals throughout this process.
Table H: Complete stage-wise correctness transitions for reference seed 47. Epoch 120 is the final analysis reconstruction. The event total counts transitions between consecutive checkpoints and can count the same identity more than once. The endpoint comparison counts each identity once by comparing epochs 20 and 120.
| Epoch interval or comparison | Correct at start | Correct at end | Wrong → Correct | Correct → Wrong | Net change |
|---|---|---|---|---|---|
| 20 → 40 | 307 | 310 | 8 | 5 | +3 |
| 40 → 60 | 310 | 308 | 4 | 6 | −2 |
| 60 → 80 | 308 | 313 | 10 | 5 | +5 |
| 80 → 100 | 313 | 311 | 5 | 7 | −2 |
| 100 → 120 | 311 | 316 | 9 | 4 | +5 |
| Consecutive-stage event total | — | — | 36 | 27 | +9 |
| Epoch-20 vs epoch-120 endpoints | 307 | 316 | 26 | 17 | +9 |
Table H records 36 correction and 27 reversal events, with corrections in every interval and the largest net gains of five during epochs 60–80 and 100–120. Figure 3(b) (page 7) and Table 6 (page 24) track the resulting correspondence accuracy and reliable set: correspondence accuracy improves overall from 77.7% to 80.0%, while reliable coverage expands from 67.3% to 76.5% and precision stays above 91%. Table 8(c) (page 24) records routing transitions, including 20 initially reliable relations being downgraded and 56 initially non-reliable relations becoming reliable. Thus, SUAL adapts supervision to current uncertainty, while CRCE revises the matches and routing as representations evolve, supporting correction from early through late training.
Response to Reviewer kFYm (1/5): Distinctive Contribution of ReCUE
A1: ReCUE's distinctive contribution is a revisable supervision mechanism: global assignment ambiguity determines the learning target, and representation learning changes the assignments from which that target is reconstructed. Recent ASM [2] uses momentum soft-label updates, and Multi-Memory Matching [3] uses soft cluster-level alignment. Section 3.3 (pages 4 and 5) defines ReCUE support as the fraction of retained lowest-cost feasible assignments containing an edge and uses it to choose hard, consensus-soft, or deferred supervision. Section 3.4 (pages 5 and 6) reconstructs prototypes, costs, hypotheses, support, and routing from the updated representation, allowing reliable edges to be replaced or downgraded. Table 2(c) (page 7) tests this coupling by crossing three uncertainty representations with Locked and Full updating, extended to five seeds in Table G.
Table G: Per-seed representation–update comparison on SYSU-MM01 All Search. Locked preserves reliable edges. Full reconstructs the complete correspondence state. Each pair of columns compares the two update rules under the same uncertainty representation.
Proposition 7 of Appendix A.1 (page 16) establishes that the consensus loss and logit gradient average their hard-target counterparts over the retained hypotheses. Proposition 8 (pages 16 and 17) shows that retaining target mass beyond the selected partner cannot generally be represented by scalar weighting of a hard-target loss. In Table G, uniform Top-K gains 0.32–0.56 percentage points in mAP with Full updating, with both retrieval metrics improving in every run. The larger gains with uniform Top-K relative to the Local and Sinkhorn controls support the interaction discussed in Section 3.7 (page 8), where the benefit of reconstruction depends on the uncertainty representation used for supervision. Thus, the distinctive contribution is the coupling of globally informed supervision targets with complete correspondence revision, supported by consistent Full–Locked gains for uniform Top-K.
[2] Pang et al. “Augmented and Softened Matching for Unsupervised Visible-Infrared Person Re-Identification.” ICCV, 2025. [3] Shi et al. “Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification.” ECCV, 2024.