- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates generalizable object re-identification (ReID), focusing on how representations learned during adaptation transfer to unseen object categories. The authors identify a phenomenon called Progressive Feature Repurposing (PFR), where continued adaptation improves discrimination on seen categories but does not necessarily improve transfer performance, while earlier representations may retain useful information. Based on this observation, they propose ReState, which consists of three components: (1) Trajectory Residual Routing (TRR), which combines historical adaptation states through input- and dimension-dependent routing; (2) Dual-Scale Trajectory Regularization (DTR), which regulates gradient updates and accumulated adaptation; and (3) Dual-Space Relation Perturbation (DSRP), which modifies classification and metric-learning supervision to reduce over-specialization to seen identities.
The method is evaluated on ten retrieval tasks covering different object categories and distribution shifts. The reported results show improvements over a strong adaptation baseline across all seven category-shifted tasks and competitive performance against existing ReID methods. Additional experiments analyze adaptation trajectories, historical feature restoration, component contributions, and routing strategies to support the proposed approach.
Critical strengths
- The paper provides an interesting analysis of how representations change during ReID adaptation. The results show that improvements on seen categories do not always translate into better performance on unseen categories, and that earlier representations can still be useful. The historical restoration experiments and query-level analysis provide evidence for this observation, giving a reasonable motivation for retaining intermediate adaptation states.
- The experimental evaluation covers a wide range of object categories and retrieval settings. ReState improves over the GVA baseline on all seven category-shifted tasks and achieves competitive results against existing ReID methods. The component ablations and comparisons with simpler routing strategies also help identify where the improvements come from. These results suggest that reusing adaptation history can be useful for cross-category generalization, even though the necessity of the full routing mechanism remains unclear.
Critical weaknesses
- The novelty of Progressive Feature Repurposing (PFR) is not sufficiently established. The observation that continued adaptation improves source-task performance while potentially reducing transferability is closely related to existing findings on feature distortion and over-specialization during fine-tuning. Although the authors provide additional analyses of feature subspaces and historical restoration, it remains unclear what fundamentally distinguishes PFR from these known phenomena. This weakens the significance of the paper's central motivation.
- The necessity of the proposed Trajectory Residual Routing (TRR) mechanism is questionable. Appendix Table 9 shows that TRR does not consistently outperform simpler alternatives, including static coordinate-wise routing and input-conditioned state-wise routing. For example, on University D→S, static coordinate-wise routing achieves 28.4 mAP, compared with 27.8 for TRR. Similar results appear on CUTE and ILIAS. These findings suggest that the additional complexity of combining input-dependent and dimension-wise routing may not be justified by its performance.
- The contribution of historical state reuse is not clearly separated from trajectory regularization. In Table 3, DTR alone provides larger improvements than TRR alone on several category-shifted tasks. For instance, on University D→S, DTR achieves 28.9 mAP, while TRR reaches 27.8. This raises the question of whether the main improvements come primarily from controlling the adaptation process rather than reusing historical representations. Although the full method performs well, the experiments do not fully establish the independent necessity of its components.
- The computational cost of ReState is not adequately evaluated. At inference, TRR requires extracting representations from multiple historical adaptation states and the pretrained anchor. Sharing the frozen backbone parameters reduces storage requirements but does not eliminate the cost of multiple forward passes. The paper does not provide measured inference latency, FLOPs, or a performance-cost comparison with single-state alternatives. Given that the routing improvements over simpler variants are relatively small, this additional inference overhead is important when assessing the practical value of the method.
Questions for authors
- How does Progressive Feature Repurposing (PFR) differ from previously studied feature distortion or loss of transferability during fine-tuning? What evidence supports treating PFR as a distinct phenomenon rather than another manifestation of over-specialization?
- In Appendix Table 9, TRR performs similarly to or worse than simpler routing strategies on several datasets. Could the authors explain why both input-dependent and dimension-wise routing are necessary? Under what conditions does their combination provide a clear advantage?
- Table 3 shows that DTR alone contributes more than TRR alone on several category-shifted tasks. Could the authors provide a clearer analysis of how much the final improvement comes from historical state reuse versus trajectory regularization? In particular, how does TRR perform when applied to states trained with DTR but without DSRP?
- Could the authors report the actual inference latency, FLOPs, and memory overhead of ReState compared with a single adapted encoder? How does retrieval performance change when fewer historical states are retained? This would help determine whether the additional computation is justified by the reported gains.
2 comments
Response to Reviewer 6mVY (Part 2)
5. Does retaining fewer historical states reduce retrieval accuracy? (Q4)
Response: In the existing cache-interval experiment, fewer states instead improve mean mAP while reducing measured latency.
| Cache interval | State count | Four-task mean mAP | Single-image latency ms |
|---|---|---|---|
| 500 (default) | 10 | 40.10 | 92.61 |
| 600 | 8 | 40.48 | 76.94 |
| 800 | 6 | 40.81 | 62.20 |
| 900 | 5 | 41.12 | 54.96 |
This experiment consistently evaluates PetFace, University D→S, CUTE, and ILIAS. The mAP values come from independent GVA+TRR training runs with different cache intervals; latency comes from structure-level measurements on one A800 with batch=1 (see Appendix B.6, Fig. 6 for the parameter sweep). Fixed intervals do not guarantee selection of the most suitable historical states, and improved state selection may further increase efficiency.
References
[1] Kumar, A., Raghunathan, A., Jones, R., Ma, T. and Liang, P. "Fine-Tuning Can Distort Pretrained Features and Underperform Out-of-Distribution." ICLR, 2022.
[2] Mukhoti, J., Gal, Y., Torr, P.H. and Dokania, P.K. "Fine-Tuning Can Cripple Your Foundation Model; Preserving Features May Be the Solution." Transactions on Machine Learning Research (TMLR), 2024.
[3] Andreassen, A., Bahri, Y., Neyshabur, B. and Roelofs, R. "The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning." Transactions on Machine Learning Research (TMLR), 2022.
Response to Reviewer 6mVY (Part 1)
We thank the reviewer for their comments and suggestions.
1. How does PFR substantively differ from existing studies of fine-tuning distortion? (W1/Q1)
Response: PFR reveals a mismatch across training stages: while source identity discrimination continues to improve, earlier features can still improve the final model's instance retrieval on unseen categories. Existing studies of fine-tuning distortion [1,2] and OOD trajectories [3] do not directly answer whether these historical features can improve identity matching after training is complete. We examine this through a fixed early discriminative subspace, refitted category readouts, and restoration of early features.
We consistently report four representative tasks with established restoration results, covering animal, cross-view, and multi-category retrieval. The complete positive and negative restoration effects and random-restoration controls are in Appendix A.6, Table 6; category readouts are in Appendix A.5, Table 5. ReState's design advantage is to retain and reuse these historical states without having to choose a stopping point in advance for unseen targets.
2. Does input-dependent, dimension-wise TRR outperform simple combinations? (W2/Q2)
Response: TRR computes historical-state combination weights for each feature dimension based on changes in the input image's features; with the same state pool, it outperforms fixed feature averaging using the same number of encoder passes. The added same-trajectory experiment shows all three category-aligned tasks, University D→S, and all three multi-category tasks:
Equal-weight averaging is slightly better on the three category-aligned tasks; TRR leads on the four displayed unseen-category tasks and on all seven unseen-category tasks in the complete evaluation. This demonstrates the value of learned combination weights for category transfer; the advantage cannot be explained solely by additional encoder passes. Other simplified routing forms are reported in Appendix B.7, Table 9.
3. Given DTR's gains, how much additional contribution does TRR provide? (W3/Q3)
Response: TRR combined with DTR outperforms DTR alone on all five tasks in the component experiment. DTR changes state formation through gradient reversal and LoRA shrinkage, while TRR reuses features from these stages, making their roles complementary.
We report all five component-experiment tasks from Table 3 and Appendix B.7, Table 10; this is a different experiment from the repeated training across ten tasks. State formation and reuse can jointly improve unseen-category retrieval.
4. Does multi-state inference incur excessive cost for limited gains? (W4/Q4)
Response: The actual overhead can be substantially reduced through fewer states and multi-GPU parallelism; full ReState raises the ten-task mean mAP from GVA's 26.65 to 29.08, with gains on all seven unseen-category tasks. State features can be encoded independently and then combined. Measurements on A800 GPUs with batch=512 are:
On one GPU, five-state throughput is only 11.2% lower than OG-ReID (CVPR 2026), with 12.7% higher batch latency and 75.2% lower peak memory; two GPUs further increase throughput. The default ten-state configuration is not the lowest-cost option. TRR also outperforms equal-weight averaging with the same state pool (Question 2), showing that effective feature combination contributes to the accuracy gains. Future work can further investigate which historical states are worth retaining.