acceptodds
Under review as a conference paper at ICLR 2027

Correspondence-Representation Co-Evolution for Weakly Supervised Visible-Infrared Person Re-Identification

Abstract

Weakly supervised visible-infrared person re-identification learns from independently labeled visible and infrared images of the same training identities, without access to their cross-modal correspondences. When provisional matches supervise representation learning, early matching errors can distort the representation and be reinforced in subsequent correspondence inference. In this paper, we propose ReCUE, a correspondence-representation co-evolution framework that uses agreement among competing global assignments to guide supervision while keeping every correspondence revisable. Specifically, Global Correspondence Hypothesis Inference (GCHI) enumerates the K lowest-cost feasible one-to-one assignments and measures each candidate match's support by its frequency across these assignments. Structural Uncertainty-Aware Learning (SUAL) translates this global support into differentiated supervision: highly supported matches receive hard targets, ambiguous matches retain soft targets over competing counterparts, and weakly supported matches receive no correspondence-derived positive supervision. As the representation evolves, Correspondence-Representation Co-Evolution (CRCE) periodically reconstructs the complete correspondence state and associated supervision, allowing previously accepted matches to be reassigned or downgraded. Experiments on SYSU-MM01 and LLCM validate the effectiveness of ReCUE, which achieves 73.0%/69.0% Rank-1/mAP on SYSU-MM01 All Search and 57.0%/60.5% on LLCM VIS→IR. The code will be available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visible-Infrared Person Re-identification, Weakly Supervised Learning, Cross-modal Correspondence, Uncertainty-aware Learning, Ranked Assignment, Correspondence Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kFYmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies weakly supervised visible-infrared person re-identification (WS-VIReID), where identity labels are available within each modality but cross-modal identity correspondences are unavailable during training. To address this issue, the authors propose ReCUE, a correspondence-representation co-evolution framework consisting of three components. First, Global Correspondence Hypothesis Inference (GCHI) estimates correspondence reliability by considering the agreement among Top-K globally feasible one-to-one assignments. Second, Structural Uncertainty-Aware Learning (SUAL) introduces differentiated supervision strategies, including hard supervision, soft supervision, and deferred supervision, according to correspondence reliability. Third, Correspondence-Representation Co-Evolution (CRCE) periodically reconstructs cross-modal correspondences, allowing previously estimated matches to be revised as the representation improves. Experiments on SYSU-MM01 and LLCM demonstrate competitive retrieval performance.

Critical strengths
  1. The paper provides useful insights into incorporating structured correspondence uncertainty into cross-modal representation learning. In particular, the combination of global assignment consensus and revisable supervision offers an interesting perspective on mitigating error reinforcement in weakly supervised learning.

  2. The experimental results demonstrate competitive performance across multiple benchmarks. These experiments strengthen the empirical evidence supporting the proposed framework.

Critical weaknesses
  1. The proposed framework builds upon several established ideas, including global assignment, Top-K matching hypotheses, uncertainty-aware supervision, and alternating optimization. Although their integration is meaningful, the paper could further clarify the distinctive contribution of the proposed combination.

  2. A more explicit discussion of how structured assignment uncertainty provides information beyond conventional confidence estimation would help strengthen the novelty claims.

Questions for authors
  1. Could the authors provide some additional intuition on why consensus among globally feasible assignments is particularly beneficial for cross-modal supervision compared with conventional local confidence estimation?

  2. The correspondence reconstruction strategy appears effective in correcting early matching errors. Could the authors further elaborate on how SUAL and CRCE complement each other at different stages of training?

  3. Could the authors briefly report the practical computational overhead of GCHI on SYSU-MM01 and LLCM, such as the approximate time required for each correspondence refresh? This would help readers better understand the efficiency of the proposed framework.

  4. The paper already includes extensive experimental comparisons. Could the authors additionally comment on whether the reported improvements remain stable across different independent training seeds?

5 comments
AuthorsOpenReview ↗

Response to Reviewer kFYm (1/5): Distinctive Contribution of ReCUE

Weakness 1: The proposed framework builds upon several established ideas, including global assignment, Top-K matching hypotheses, uncertainty-aware supervision, and alternating optimization. Although their integration is meaningful, the paper could further clarify the distinctive contribution of the proposed combination.

A1: ReCUE's distinctive contribution is a revisable supervision mechanism: global assignment ambiguity determines the learning target, and representation learning changes the assignments from which that target is reconstructed. Recent ASM [2] uses momentum soft-label updates, and Multi-Memory Matching [3] uses soft cluster-level alignment. Section 3.3 (pages 4 and 5) defines ReCUE support as the fraction of retained lowest-cost feasible assignments containing an edge and uses it to choose hard, consensus-soft, or deferred supervision. Section 3.4 (pages 5 and 6) reconstructs prototypes, costs, hypotheses, support, and routing from the updated representation, allowing reliable edges to be replaced or downgraded. Table 2(c) (page 7) tests this coupling by crossing three uncertainty representations with Locked and Full updating, extended to five seeds in Table G.

Table G: Per-seed representation–update comparison on SYSU-MM01 All Search. Locked preserves reliable edges. Full reconstructs the complete correspondence state. Each pair of columns compares the two update rules under the same uncertainty representation.

RepresentationTraining seedLocked Rank-1 (%)Full Rank-1 (%)Locked mAP (%)Full mAP (%)
Local1371.9871.8767.8767.59
Local2972.4172.5868.1568.14
Local4772.3472.3268.1068.04
Local7172.6572.6968.4068.43
Local10372.3272.1467.9868.00
Sinkhorn1372.2772.2168.1568.09
Sinkhorn2972.5572.7068.4668.56
Sinkhorn4772.4072.5468.3268.34
Sinkhorn7172.5872.8368.4368.60
Sinkhorn10372.2072.4268.2468.11
Uniform Top-K1371.9672.6068.1968.51
Uniform Top-K2972.7173.0868.7969.29
Uniform Top-K4772.6073.0068.5069.00
Uniform Top-K7173.0573.4869.0769.63
Uniform Top-K10372.1872.6468.0568.47

Proposition 7 of Appendix A.1 (page 16) establishes that the consensus loss and logit gradient average their hard-target counterparts over the retained hypotheses. Proposition 8 (pages 16 and 17) shows that retaining target mass beyond the selected partner cannot generally be represented by scalar weighting of a hard-target loss. In Table G, uniform Top-K gains 0.32–0.56 percentage points in mAP with Full updating, with both retrieval metrics improving in every run. The larger gains with uniform Top-K relative to the Local and Sinkhorn controls support the interaction discussed in Section 3.7 (page 8), where the benefit of reconstruction depends on the uncertainty representation used for supervision. Thus, the distinctive contribution is the coupling of globally informed supervision targets with complete correspondence revision, supported by consistent Full–Locked gains for uniform Top-K.

[2] Pang et al. “Augmented and Softened Matching for Unsupervised Visible-Infrared Person Re-Identification.” ICCV, 2025. [3] Shi et al. “Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification.” ECCV, 2024.

AuthorsOpenReview ↗

Response to Reviewer kFYm (2/5): Global Consensus versus Local Confidence

Weakness 2: A more explicit discussion of how structured assignment uncertainty provides information beyond conventional confidence estimation would help strengthen the novelty claims.

Question 1: Could the authors provide some additional intuition on why consensus among globally feasible assignments is particularly beneficial for cross-modal supervision compared with conventional local confidence estimation?

A2: Global consensus measures how consistently a match survives competition for exclusive partners. A pair can have high local similarity while another identity competes for the same partner, and feasible assignments resolve these conflicts jointly. TokenMatcher [4] aggregates similarities across token embedding spaces, while ReCUE measures agreement among discrete feasible assignments in Equation (5), Section 3.3 (page 4). Proposition 3 of Appendix A.1 (page 15) constructs two problems with the same selected-edge cost and unique best assignment but different support among their two best assignments, demonstrating information beyond the local score.

Table 12 (page 26) compares reliability scores on the same 395 initial selected edges, treating incorrect matches as the positive class. Uniform Top-K achieves AUROC/AUPRC of 0.803/0.520 versus 0.688/0.407 for local cost, and reliable-set precision of 91.7% versus 84.2% at the same 67.3% coverage. Table 13 (page 26) compares precision across coverage levels and shows uniform Top-K leading at the default, 80%, and 90% coverage. This connects its error-detection advantage to the coverage range used for reliable supervision. Thus, structured consensus exposes global partner competition and provides stronger error detection and cleaner reliable supervision than local confidence at the operating coverage used by ReCUE.

[4] Wang et al. “TokenMatcher: Diverse Tokens Matching for Unsupervised Visible-Infrared Person Re-Identification.” AAAI, 2025.

AuthorsOpenReview ↗

Response to Reviewer kFYm (3/5): Complementary Roles of SUAL and CRCE

Question 2: The correspondence reconstruction strategy appears effective in correcting early matching errors. Could the authors further elaborate on how SUAL and CRCE complement each other at different stages of training?

A3: Structural Uncertainty-Aware Learning determines supervision within each stage, and Correspondence-Representation Co-Evolution revises the correspondence state between stages. Equation (7) in Section 3.3 (page 5) assigns hard targets to reliable matches, soft targets to ambiguous matches, and intra-modal learning with deferred correspondence supervision to unreliable matches. Equation (9) in Section 3.4 (page 6) rebuilds prototypes, costs, hypotheses, support, and routing after representation updates, allowing matches to receive stronger supervision or be downgraded or replaced. Table H separates corrections from reversals throughout this process.

Table H: Complete stage-wise correctness transitions for reference seed 47. Epoch 120 is the final analysis reconstruction. The event total counts transitions between consecutive checkpoints and can count the same identity more than once. The endpoint comparison counts each identity once by comparing epochs 20 and 120.

Epoch interval or comparisonCorrect at startCorrect at endWrong → CorrectCorrect → WrongNet change
20 → 4030731085+3
40 → 6031030846−2
60 → 80308313105+5
80 → 10031331157−2
100 → 12031131694+5
Consecutive-stage event total——3627+9
Epoch-20 vs epoch-120 endpoints3073162617+9

Table H records 36 correction and 27 reversal events, with corrections in every interval and the largest net gains of five during epochs 60–80 and 100–120. Figure 3(b) (page 7) and Table 6 (page 24) track the resulting correspondence accuracy and reliable set: correspondence accuracy improves overall from 77.7% to 80.0%, while reliable coverage expands from 67.3% to 76.5% and precision stays above 91%. Table 8(c) (page 24) records routing transitions, including 20 initially reliable relations being downgraded and 56 initially non-reliable relations becoming reliable. Thus, SUAL adapts supervision to current uncertainty, while CRCE revises the matches and routing as representations evolve, supporting correction from early through late training.

Reviewer_M6VDOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Training images have separate identity labels within each modality, but their cross-modal identity links are unknown, early matching errors can affect feature learning and lead to further incorrect matches. Therefore, this paper proposes ReCUE for weakly supervised visible-infrared person re-identification. ReCUE includes three components: 1) Global Correspondence Hypothesis Inference finds the K lowest-cost global one-to-one assignments and counts how often each match appears. 2)Structural Uncertainty-Aware Learning uses this support to choose different training targets. 3) Correspondence-Representation Co-Evolution regularly updates all assignments and training targets as the features change.

Critical strengths
  1. The paper explains how early matching errors can affect feature learning and lead to further incorrect matches, keeping earlier matches open to revision directly addresses this problem.

  2. The method checks each match across several complete one-to-one assignments. This considers competition between identities and provides more information than using only the best assignment.

  3. The flexible supervision and updates is reasonable in this topic. Reliable, ambiguous, and unreliable matches receive different training signals. Soft targets keep several possible matches, while deferred supervision avoids forcing uncertain positive pairs.

Critical weaknesses
  1. Some routing comparisons change both the target type and which matches receive supervision. The separate benefit of soft targets is therefore not fully clear.

  2. Finding multiple global assignments and rebuilding correspondences add computation during training. These steps have a clear purpose, but the paper provides limited information about their actual running time. The balance between the accuracy gains and the extra training cost is therefore difficult to assess.

  3. Table 2 uses the general name “Baseline,” and its results differ from HECCL in Table 1. The main text should identify this baseline and state which results are reproduced and which are taken from previous work.

Questions for authors
  1. soft targets could be tested more directly. Table 2(d) provides useful routing ablations. However, the no soft variant also removes positive supervision for ambiguous matches. So, what is the performance using the same loss weight and deferral rule?

  2. The training cost is not clear enough. GCHI finds several global assignments, while CRCE rebuilds them during training. Please report the time for one correspondence update and the total training time. A comparison with Top-1 matching or Sinkhorn on the same hardware would help show the extra cost.

  3. It will be better with a deeper analyze about the remaining high-support errors. Table 3(b) shows that only 3 of the 22 wrong matches first marked as reliable are corrected. The authors could examine a few of these cases and show their support scores across training stages. This would help explain why these wrong matches receive high support and remain difficult to correct after later updates.

5 comments
AuthorsOpenReview ↗

Response to Reviewer M6VD (1/4): Controlled Soft–Hard Comparison

Weakness 1: Some routing comparisons change both the target type and which matches receive supervision. The separate benefit of soft targets is therefore not fully clear.

Question 1: soft targets could be tested more directly. Table 2(d) provides useful routing ablations. However, the no soft variant also removes positive supervision for ambiguous matches. So, what is the performance using the same loss weight and deferral rule?

A1: Following your suggestion, we compared soft and hard targets under matched supervision settings in Table D. The no-soft variant in Table 2(d) (page 7) defers ambiguous pairs, as specified in Appendix C.1 (pages 25 and 26). The new Controlled Hard comparison keeps these matches supervised with λw=0.25 and replaces only their consensus target with the best-assignment one-hot target. Reliable matches retain hard supervision with λs=1, and unreliable matches remain deferred. The overall trend favors consensus soft targets in both Rank-1 and mAP.

Table D: Complete per-seed soft–hard comparison on SYSU-MM01 All Search. Only the ambiguous target changes. Both configurations supervise reliable and ambiguous matches and defer unreliable matches, using the same loss weights.

Training seedFull soft Rank-1 (%)Controlled Hard Rank-1 (%)Full soft mAP (%)Controlled Hard mAP (%)
1372.6072.4668.5168.41
2973.0872.8269.2968.83
4773.0073.1269.0069.01
7173.4873.0369.6369.21
10372.6472.4668.4768.29

Recent ASM [2] generates soft labels through momentum updates across epochs, while ReCUE forms targets from agreement among current globally feasible hypotheses. For an ambiguous match, a wrong hard target assigns all mass to the selected partner even when another retained assignment contains the correct alternative. Consensus targets allow that alternative to receive supervision before it becomes the selected partner. Proposition 5 of Appendix A.1 (pages 15 and 16) and Corollary 5.1 (page 16) quantify this effect: at the same prediction, temperature, and loss weight, positive target mass on the correct partner reduces the ℓ1\ell_1 logit-gradient discrepancy from correct-target supervision relative to a wrong hard target. Thus, the matched comparison favors consensus soft supervision overall, with a mechanism that preserves learning toward correct alternatives.

[2] Pang et al. “Augmented and Softened Matching for Unsupervised Visible-Infrared Person Re-Identification.” ICCV, 2025.

AuthorsOpenReview ↗

Response to Reviewer M6VD (2/4): Computational Efficiency

Weakness 2: Finding multiple global assignments and rebuilding correspondences add computation during training. These steps have a clear purpose, but the paper provides limited information about their actual running time. The balance between the accuracy gains and the extra training cost is therefore difficult to assess.

Question 2: The training cost is not clear enough. GCHI finds several global assignments, while CRCE rebuilds them during training. Please report the time for one correspondence update and the total training time. A comparison with Top-1 matching or Sinkhorn on the same hardware would help show the extra cost.

A2: We compared the reproduced baseline [1], Top-1 assignment, Sinkhorn, and ReCUE on the same NVIDIA RTX 4090 and Intel Core i9-13900K under the 120-epoch protocol. Table E separates assignment inference, complete correspondence updates, and total training time. Section 3.4 (page 6) specifies that each reconstructed correspondence state supervises a twenty-epoch stage, and Algorithm 1 in Appendix B.1 (page 20) separates training from the final diagnostic reconstruction. The reported training times include all five training reconstructions, with the final analysis reconstruction timed separately.

Table E: Hardware-matched timing comparison. Solver time is included in the complete refresh. Relative training time is normalized to the reproduced baseline [1].

DatasetMethodSolver (milliseconds)Refresh (seconds)Training (hours)Relative training time
SYSU-MM01Reproduced baseline [1]1.4420.293.9731.0000×
SYSU-MM01Top-1 assignment7.0520.383.9941.0053×
SYSU-MM01Sinkhorn2.6020.504.0071.0086×
SYSU-MM01ReCUE, K=10333.9420.874.0311.0146×
LLCMReproduced baseline [1]2.7528.956.1651.0000×
LLCMTop-1 assignment32.5829.126.1951.0049×
LLCMSinkhorn7.2629.346.2181.0086×
LLCMReCUE, K=101914.5431.296.2711.0172×

Table E shows complete ReCUE updates of 20.87 seconds on SYSU-MM01 and 31.29 seconds on LLCM. Across the full training schedule, ReCUE takes approximately 2.2 and 4.6 additional minutes compared with Top-1, respectively, and remains close to Sinkhorn in total training time. Table 2(a) (page 7) reports the corresponding SYSU-MM01 retrieval improvement from 66.3% mAP for the reproduced baseline [1] to 69.0% for Full ReCUE. Thus, ReCUE's measured retrieval gain is accompanied by total training overhead of 1.46% on SYSU-MM01 and 1.72% on LLCM, with end-to-end time close to both matching controls.

[1] Zhang et al. “Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning.” ICCV, 2025.

AuthorsOpenReview ↗

Response to Reviewer M6VD (3/4): Baseline Identification and Result Sources

Weakness 3: Table 2 uses the general name “Baseline,” and its results differ from HECCL in Table 1. The main text should identify this baseline and state which results are reproduced and which are taken from previous work.

A3: Thank you for pointing out this ambiguity. We have clarified the baseline name and result sources in the latest PDF. Table 1 (page 6) reports the published HECCL [1] results of 70.4% Rank-1 and 66.6% mAP. Table 2(a) (page 7) reports 70.7% Rank-1 and 66.3% mAP for the reproduced baseline [1], which provides the starting point for the component comparisons.

Appendix B.1 (page 20) documents the HECCL reimplementation and specifies the same 20-epoch first stage and 100-epoch second stage for the baseline and proposed variants. Thus, the two tables distinguish published results from a controlled reproduction, and all component comparisons in Table 2 use the common reproduction protocol.

[1] Zhang et al. “Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning.” ICCV, 2025.

Reviewer_4zAcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper considers weakly supervised visible-infrared person re-identification, where identities are labeled separately in each modality but their cross-modal matches are unknown. ReCUE aims to prevent early matching errors from being reinforced during training. GCHI generates the K lowest-cost one-to-one assignments and measures how often each match appears. SUAL then uses this frequency to choose hard or soft supervision, or to withhold cross-modal supervision when support is weak. CRCE periodically recomputes all correspondences so that earlier decisions can be corrected. Results on SYSU-MM01 and LLCM improve over the reported weakly supervised baseline. Ablations and controlled experiments further examine the supervision choices, uncertainty estimates, and revision of incorrect matches.

Critical strengths
  1. The paper identifies a concrete failure mode in weakly supervised VI-ReID: an incorrect cross-modal match can distort feature learning and become harder to correct later. ReCUE addresses this by keeping correspondence decisions revisable throughout training. This gives the method a clear motivation and connects the matching procedure directly to the quality of the learned representation.
  2. GCHI considers several feasible one-to-one assignments before deciding how much to trust a match. This is useful because a plausible pair may conflict with other identity assignments. Measuring how often an edge appears across competing assignments gives the model information about this global competition that an isolated similarity score does not provide.
  3. The retrieval gains are consistent across SYSU-MM01 and both LLCM directions. On SYSU-MM01 All Search, ReCUE improves Rank-1/mAP from HECCL’s 70.4/66.6 to 73.0/69.0, while also reporting improvements under Indoor Search. Results across datasets and retrieval directions give broader support than an improvement on a single evaluation setting.
Critical weaknesses
  1. The computational cost of repeated Top-K assignment is not measured empirically. Appendix B.2 gives the theoretical complexity, but does not report correspondence-refresh time, memory usage, or total training overhead. Since each Hungarian subproblem costs O(N3)O(N^3) and Murty enumeration requires additional solves, runtime measurements at larger identity counts would help establish the method’s scalability.
  2. The evaluation assumes that both modalities contain exactly the same training identities, allowing a complete one-to-one assignment. Appendix B.2 describes a partial-overlap extension, but no experiments test it. It remains unclear how well the support estimates and supervision rules work when some identities appear in only one modality, especially when the overlap size or rejection cost is misspecified.
Questions for authors
  1. I would like to see experiments with partial identity overlap between modalities, showing how performance changes with the overlap ratio and errors in the assumed number of shared identities.
  2. How much time and memory does each correspondence refresh require? Please report the total training overhead and how the assignment cost grows with the number of identities.
  3. How stable are the Full–Locked gains across independent training runs? Table 3 also shows that initially wrong reliable matches are harder to correct. Does this pattern persist across runs, and what distinguishes the high-confidence errors that are eventually corrected from those that remain wrong?
  4. Do the same routing thresholds and refresh interval work on both datasets without retuning? Could you clarify how these settings are selected without using cross-modal correspondence labels or test performance?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 4zAc (1/4): Computational Efficiency

Weakness 1: The computational cost of repeated Top-K assignment is not measured empirically. Appendix B.2 gives the theoretical complexity, but does not report correspondence-refresh time, memory usage, or total training overhead. Since each Hungarian subproblem costs (O(N^3)) and Murty enumeration requires additional solves, runtime measurements at larger identity counts would help establish the method’s scalability.

Question 2: How much time and memory does each correspondence refresh require? Please report the total training overhead and how the assignment cost grows with the number of identities.

A1: Following your suggestion, Table A reports assignment inference, complete correspondence refreshes, total training time, and peak training graphics processor memory under the same 120-epoch protocol on an NVIDIA RTX 4090 and Intel Core i9-13900K. Section 3.4 (page 6) describes five twenty-epoch training stages after warm-up, with correspondence reconstructed at stage boundaries. Algorithm 1 in Appendix B.1 (page 20) specifies that reconstruction recomputes prototypes, assignment costs, hypotheses, support, and supervision categories. Training time includes the five training reconstructions, while the final reconstruction at epoch 120 is used for analysis and timed separately.

Table A: Computational efficiency on SYSU-MM01 (395 training identities) and LLCM (713 training identities). Solver time is included in refresh time. Relative training time is normalized to the reproduced baseline [1]. Peak memory refers to graphics processor memory during training.

DatasetMethodSolver (milliseconds)Refresh (seconds)Training (hours)Relative training timePeak training memory (gibibytes)
SYSU-MM01Reproduced baseline [1]1.4420.293.9731.0000×8.162
SYSU-MM01Top-1 assignment7.0520.383.9941.0053×8.186
SYSU-MM01Sinkhorn2.6020.504.0071.0086×8.252
SYSU-MM01ReCUE, K=10333.9420.874.0311.0146×8.275
LLCMReproduced baseline [1]2.7528.956.1651.0000×8.410
LLCMTop-1 assignment32.5829.126.1951.0049×8.443
LLCMSinkhorn7.2629.346.2181.0086×8.572
LLCMReCUE, K=101914.5431.296.2711.0172×8.604

Appendix B.2 (page 20) specifies O(N²) cost-matrix storage and O(N³) computation per Hungarian subproblem, with additional constrained solves for Top-K enumeration. Between the two benchmark sizes of 395 and 713 identities in Table A, ReCUE solver time increases by about 5.7 times, while complete-refresh time increases by about 1.5 times, from 20.87 to 31.29 seconds. Peak training memory increases from the reproduced baseline's 8.162 to 8.275 gibibytes on SYSU-MM01 and from 8.410 to 8.604 gibibytes on LLCM. Thus, complete refreshes remain practical at both benchmark sizes, with total training overhead of 1.46% and 1.72% and small increases in peak training memory.

[1] Zhang et al. “Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning.” ICCV, 2025.

AuthorsOpenReview ↗

Response to Reviewer 4zAc (2/4): Evaluation Protocol

Weakness 2: The evaluation assumes that both modalities contain exactly the same training identities, allowing a complete one-to-one assignment. Appendix B.2 describes a partial-overlap extension, but no experiments test it. It remains unclear how well the support estimates and supervision rules work when some identities appear in only one modality, especially when the overlap size or rejection cost is misspecified.

Question 1: I would like to see experiments with partial identity overlap between modalities, showing how performance changes with the overlap ratio and errors in the assumed number of shared identities.

A2: Our work follows the weakly supervised visible-infrared person re-identification protocol introduced by HECCL [1]: identity labels are available within each modality, while cross-modal identity correspondences are unknown. Section 3.1 (page 3) specifies the same N training identities with independently assigned labels in both modalities. This shared-identity prior makes the unknown mapping a permutation, as formalized in Equation (1). The mapping is withheld during training and model selection, and retrieval is evaluated on disjoint test identities.

The two problems differ in their focus. Under the shared-identity protocol, the question is which identities correspond across modalities. In partial-overlap matching, the question is which identities have cross-modal counterparts. Accordingly, ReCUE studies correspondence recovery under the established shared-identity protocol adopted by HECCL [1], and its empirical conclusions concern this setting.

[1] Zhang et al. “Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning.” ICCV, 2025.

AuthorsOpenReview ↗

Response to Reviewer 4zAc (3/4): Multi-Seed Stability and Correspondence Recovery

Question 3: How stable are the Full–Locked gains across independent training runs? Table 3 also shows that initially wrong reliable matches are harder to correct. Does this pattern persist across runs, and what distinguishes the high-confidence errors that are eventually corrected from those that remain wrong?

A3: Following your suggestion, we evaluated Full and Locked using five independent training seeds, with 120 epochs per run. Table B shows higher Rank-1 and mAP for Full in every run, with gains of 0.37–0.64 and 0.32–0.56 percentage points, respectively. Therefore, the benefit of revisable correspondence learning under uncertainty-aware supervision is consistent across the five tested seeds.

Table B: Per-seed Full–Locked comparison on SYSU-MM01 All Search.

Training seedLocked Rank-1 (%)Full Rank-1 (%)Locked mAP (%)Full mAP (%)
1371.9672.6068.1968.51
2972.7173.0868.7969.29
4772.6073.0068.5069.00
7173.0573.4869.0769.63
10372.1872.6468.0568.47

Table 3(b) (page 9) and Table 7(b) (page 24) report recovery by initial supervision category for reference seed 47. Table C extends this analysis to all five seeds. Initially wrong reliable matches recover at rates of 8.3%–19.0%, compared with 33.3%–36.8% for ambiguous matches and 32.3%–40.0% for unreliable matches. Thus, this category-dependent recovery pattern is consistent across all five training runs.

Table C: Per-seed recovery of initially wrong correspondences from epoch 20 to 120, grouped by initial supervision category.

Training seedInitial categoryInitially wrongCorrected at epoch 120Recovery (%)
13Reliable2428.3
13Ambiguous381436.8
13Unreliable291137.9
13Total912729.7
29Reliable21419.0
29Ambiguous361233.3
29Unreliable281139.3
29Total852731.8
47Reliable22313.6
47Ambiguous371335.1
47Unreliable291034.5
47Total882629.5
71Reliable23313.0
71Ambiguous351234.3
71Unreliable301240.0
71Total882730.7
103Reliable20210.0
103Ambiguous391333.3
103Unreliable311032.3
103Total902527.8

Among the 22 initially wrong reliable matches in reference seed 47, three matches are correct at epoch 120, five matches become correct temporarily before reverting, and 14 matches remain wrong at every observed checkpoint. The tracked trajectories distinguish recovery to the true counterpart from reassignment to another incorrect partner as support for the original wrong edge declines. Section 3.3 (pages 4 and 5) defines support as agreement among current cost-based hypotheses. Consequently, incorrect alternatives can receive high support when those hypotheses favor them. The selected partner comes from the minimum-cost global assignment, rather than the largest support in its row. Proposition 2 in Appendix A.1 (pages 14 and 15) establishes that reconstruction preserves access to alternative bijections, and Corollary 2.1 (page 15) shows that reconstruction returns the true assignment when the updated cost makes it the unique optimum. Thus, successful recovery requires the evolving global assignment to select the correct counterpart, and sustained recovery requires that selection to persist at later observed checkpoints.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.