acceptodds
Withdrawn from ICLR 2027

STaG: Human-Annotation-Free Unsupervised Domain Adaptation for Text-Based Person Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Real-world deployment of Text-Based Person Retrieval (TBPR) is bottlenecked by the absence of textual annotations in new domains. Existing Unsupervised Domain Adaptation (UDA-TBPR) methods still assume access to target-domain human-written annotations (albeit unpaired with images), limiting their practicality. We consider a more practical setting: human-annotation-free UDA-TBPR, with labeled image–text data in the source domain and annotation-free, unlabeled images in the target domain. This setting introduces two core challenges: the absence of cross-modal alignment in the target domain, and visual domain shifts from source to target. To tackle these, we propose STaG, a dual-path framework comprising Style-Agnostic Text Alignment (SATA) and Heterogeneous Graph Memory Alignment (HGMA). Specifically, SATA leverages source-domain data to decouple identity-discriminative semantics from machine-generated stylistic artifacts, establishing robust cross-modal alignment for the target domain. Meanwhile, HGMA exploits cross-domain semantic neighborhoods via a memory-based graph to learn domain-invariant visual features. Ultimately, STaG achieves state-of-the-art performance across three benchmarks under this realistic setting.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
multimodal, text-image retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sWG7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the more practical unsupervised domain adaptation settings of text-based human retrieval (TBPR), in which the source domain provides paired image-text supervision, while the target domain only contains unmarked images without any human written description. In order to solve the missing target domain text pattern and visual domain transfer, the author proposed STaG, which consists of two components: style-unknown text alignment (SATA) and heterogeneous image memory alignment (HGMA). SATA uses frozen MLLM to generate pseudo-descriptions for source images and target images, and uses descriptions written by source humans to encourage cross-modal representation with unchanged styles. HGMA maintains source image, source text and target image memory, and aggregates cross-domain and intra-domain neighbors to improve target representation, as well as residual gate and confidence perception entropy regularizer. Experiments on CUTR-PEDES, ICFG-PEDES and RSTPReid show that STaG achieves state-of-the-art performance across three benchmarks under this realistic setting

Critical strengths
  1. The paper studies a meaningful human-annotation-free UDA setting for TBPR, requiring only unlabeled target images.
  2. SATA and HGMA address complementary challenges of missing cross-modal supervision and visual domain shift.
  3. The method is evaluated across multiple datasets, transfer settings, backbones, ablations, and robustness analyses.
Critical weaknesses
  1. The adaptation protocol is insufficiently clarified. It is unclear whether only the target training split is used during adaptation and whether target annotations are involved in hyperparameter or checkpoint selection, which may weaken the human-annotation-free claim.
  2. Some methodological formulations are not fully rigorous. The SATA objective is inconsistent with the batch-based InfoNCE definition, and several details of HGMA, such as neighborhood definition, residual gating, and CAE, require clearer mathematical justification.
  3. The claimed mechanisms are not always fully supported by the evidence. In particular, the “style-agnostic” property of SATA and the “confidence-aware” behavior of CAE are only indirectly validated, and several conclusions appear stronger than the corresponding ablation gains.
  4. Experimental rigor could be improved. The paper does not report multi-run statistics, while some improvements are small; several control experiments also do not completely isolate the intended factors.
  5. Reproducibility and presentation details remain incomplete. Some hyperparameters, backbone integration details, and training procedures are under-specified, and a few numerical and figure inconsistencies should be corrected.
Questions for authors
  1. Target-domain protocol: During adaptation, do you use only the target-domain training split, or are validation/test images also involved in pseudo-text generation or training? Are any target-domain annotations used for hyperparameter or checkpoint selection?
  2. SATA optimization: How is Eq. (5) implemented in practice given that InfoNCE is defined over a mini-batch? When a source image has both a human-written description and a pseudo-description, are these treated as multiple positives, or can they become negatives to each other?
  3. HGMA training mechanism: Is the HGMA-refined target feature v~t\tilde{v}_t directly used in the target-domain contrastive objective? If HGMA is removed at inference, how is the information learned through graph aggregation transferred back to the retrieval backbone?
  4. Statistical reliability: Some ablation gains, particularly those from the residual gate and CAE, are relatively small. Could the authors provide results over multiple random seeds or otherwise clarify whether these improvements are larger than normal training variance?
Reviewer_R22yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STaG, which consists of Style-Agnostic Text Alignment and Heterogeneous Graph Memory Alignment. The problem is relevant, and the framework is reasonably comprehensive. However, the methodological novelty and the evidence supporting the proposed mechanisms remain insufficiently convincing.

Critical strengths
  1. The paper addresses a practical domain adaptation scenario where target-domain images lack human-annotated text descriptions.
  2. The experimental studies and results are relatively comprehensive.
Critical weaknesses
  1. The framework mainly combines existing techniques.The paper should clarify the methodological advances beyond these existing techniques and explain which component provides the principal innovation.
  2. Minimizing the entropy of image-to-text matching distributions does not guarantee correct matches. Incorrect candidates may also receive increasingly concentrated probabilities.
Questions for authors
  1. Using the same MLLM and prompting strategy may produce more consistent captions. The contribution of style alignment should be distinguished from the benefits of additional pseudo-caption supervision. How sensitive is final performance to the quality of the offline MLLM generator?
  2. In real deployment the target gallery can contain many more identities and far more severe domain shifts. Have you stress-tested STaG on more challenging target collections beyond the three standard benchmarks?
  3. The residual gate gtg_t and CAE loss are both confidence-based. Is there evidence that they address complementary failure modes, or do they largely overlap?
  4. The paper does not adequately establish the stability of its improvements across independent runs. Have you tried multiple random conditions to ensure the consistent of the observed gains?
Reviewer_vi9yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates unsupervised domain adaptation for text-based person retrieval when the source domain contains paired image-text data but the target domain contains only unlabeled images. The proposed STaG framework uses a frozen multimodal large language model to generate descriptions offline. SATA jointly trains on human-written and generated source descriptions as well as generated target descriptions to enhance robustness to text style. HGMA uses residual gating and an entropy regularizer to retrieve and aggregate neighbors from source image, source text, and target image memory. The overall experimental results are valuable. However, existing evidence does not yet sufficiently distinguish the novelty of the style disentanglement in this paper from related work, and the adaptation protocols of the baselines in some tables are also unclear.

Critical strengths
  1. The target domain requires no manual descriptions, which can reduce annotation costs when deploying in new scenarios and thus has practical significance. In addition, it is compared with existing UDA-TBPR methods, and the differences are explained.
  2. The experiments in this paper are relatively comprehensive and cover multiple transfer directions.
Critical weaknesses
  1. I believe the innovativeness and contribution of the SATA component in this paper remain to be verified. At the same time, the comparison between this part and existing methods for correcting large-model-generated text is not strong enough.
  2. The use of various types in this paper involves the need for different baselines, and whether these baselines use the same pseudo-text has not been consistently explained. For example, on page 7, the compared methods use the same target-domain pseudo-text generated by an MLLM for reproduction. However, in Table 1, TBPS-CLIP achieves 35.99 on ICFG→CUHK; Table 12 then uses 35.99 as the baseline and lists the actual naive pseudo-text adaptation as 38.85.
Questions for authors
  1. What data and training pipelines are used respectively for TBPS-CLIP 35.99 in Table 1, source-only 35.99 and plain pseudo-texts 38.85 in Table 12? Please provide a supervision access checklist for all baselines in the main tables.
  2. Compared with TDE of Tan et al. 2024, ordinary multi-text augmentation, and existing text style normalization schemes, what specific new capabilities or differences does SATA add?
  3. Why is "w/o CAE" in Table 4 clearly better than SATA-only? What impact does SATA have on each component of HGMA?
  4. I believe the mechanism of CAE in HGMA needs to be further explained based on the results.
  5. Some figure and text labels need to be corrected. For example, the "Source-only" column name in Figure 6 is inconsistent with the "Pseudo-Text Adaptation" in the caption. Please check the entire paper again.
  6. I believe it is necessary to explain how many times the main results and key ablations were run. Are they stable under different random seeds?
  7. Does SATA use original features or figure-corrected features? After removing HGMA, how does SATA affect the final results?

If your rebuttal clears things up for me, i'll consider raising the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.