acceptodds
Withdrawn from ICLR 2027

Text-Assisted Unsupervised Domain Adaptation for Image Regression through Semantic Re-encoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unsupervised domain adaptation for image regression transfers continuous prediction rules without target labels. Existing approaches align visual representations, but feature alignment alone need not preserve the numerical relationships required for regression. Pretrained text encoders offer an alternative representation of numerical attributes, yet target text must be constructed from uncertain predictions. We first analyze the reliability tradeoff of this representation. Deterministic target text adds no information conditional on its complete visual input, but can benefit a restricted regressor through re-encoding. A target absolute-risk bound separates source fitting, domain discrepancy, predictor compatibility, and text-construction error; squared-loss analysis further characterizes branch agreement and conditional error propagation. These results motivate retaining visual information, attending to pseudo-label reliability, and jointly supervising modality-specific and fusion predictions. Guided by these insights, we propose TCC, which encodes source labels and target pseudo-labels, trains three regression branches with a residual-informed objective, and updates the target text by fusing predictions with the preceding text. Experiments on dSprites, MPI3D, and BIWI evaluate the framework through baseline comparisons, branch ablations, and representation and parameter diagnostics. Our source code will be made publicly available.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Unsupervised Domain Adaptation, Image Regression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aGZrOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses a real and under-explored problem — continuous-output domain adaptation — and the central idea of using pretrained text encoders as a complementary representation of numerical attributes isnovel and well-motivated. The theory-to-method structure is clean, and the authors are commendably honest about what their analysis does and does not guarantee (e.g., they explicitly state that similarity is "not a calibrated probability" and that the analysis "does not guarantee ... the benefit of the additional RI regularizer"). The empirical gains are substantial on MPI3D and BIWI.

Critical strengths

1 Rigorous and honest theory. The analysis carefully separates the "information" question (Proposition 1, text adds no information given full V) from the "representation/function-class" question (a restricted regressor can still benefit). This distinction is conceptually valuable and the authors resist overclaiming throughout.

2 Clean theory-to-method mapping. Each design choice (retain visual features, fuse rather than replace; attend to pseudo-label reliability; jointly supervise modality-specific and fusion branches) is explicitly traced to a theoretical result.

Critical weaknesses
  1. Baseline and comparison fairness / contribution isolation.

    The authors themselves note (Sec. 5.2) that the benchmark comparisons "do not separately identify the contributions of pretrained features, prototype initialization, or an individual term in the risk bound." This is a significant concern: TCC combines (a) frozen pretrained visual/text encoders, (b) prototype-based target initialization, (c) three-branch supervision, and (d) the RI-Loss regularizer. It is unclear how much of the large MPI3D/BIWI gain comes from the core "text-assisted re-encoding" idea versus, e.g., the pretrained features or the pseudo-label refinement procedure. A stronger ablation would disentangle these (e.g., a version using only pretrained visual features with the same refinement loop but no text branch; a version without the RI term but with all three branches).

  2. Practical applicability and reliance on a frozen text encoder.

    The method depends on the availability of a pretrained text encoder that produces Lipschitz-continuous embeddings w.r.t. numerical values (assumed in Sec. 3). The authors acknowledge that "nearby values may tokenize differently." For real regression tasks (depth, head pose, etc.) where values are dense and high-precision, the sensitivity of the text embedding to small numerical perturbations is a central practical risk. I would like a more explicit characterization of how sensitive the final predictions are to tokenization/precision of the numeric prompts, and whether the empirical gains persist when values are not coarsely quantized.

  3. Weakness of the theory-method link for the RI term.

    The RI-Loss (from Wang et al., 2026) is imported and the authors candidly admit the analysis "neither ... guarantees the benefit of the additional RI regularizer." Given that the RI term is a nontrivial component of the method, and its statistical scope is only briefly discussed (Appendix A.7), the justification for this specific choice feels under-supported. Either provide a stronger argument, or clearly position it as an empirical engineering choice with an ablation isolating its contribution.

  4. Limited scope of datasets / task diversity.

    The evaluation covers three synthetic/semi-synthetic or small-scale benchmarks (dSprites, MPI3D, BIWI). The introduction motivates with depth estimation and head-pose prediction, yet no large-scale real regression benchmark is tested. Given the framing, a real-world regression task would substantially strengthen the paper's claim of practical relevance.

Questions for authors

Please see the weaknesses.

Reviewer_phjHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of unsupervised domain adaptation in the image regression task, which takes an input image and returns a numerical output. The authors propose TCC, which incorporates textual input into the task by converting source labels and target pseudo-labels into textual descriptions and combining their embeddings with visual features. The learning process can be described as follows: (1) From the source image and label, a set of image features, the ground-truth label are injected into a predefined template to extract the text features, and the label is formed as the source domain input to the model. (2) For the target domain, the target pseudo-labels are initialized through image–text prototype matching, followed by confidence-based sample selection to form the target domain input. (3) The model, which is described in Fig. 1, jointly supervises visual, textual, and fusion regression branches, then updates target pseudo-labels through the fusion predictor and rebuilds their textual representations for subsequent training cycles. For the experimental results on dSprites, MPI3D, and BIWI, TCC reports average MAEs of 0.052, 0.006, and 0.055, respectively. The results suggest that numerical text representations can be useful within a visual–text regression framework, although the benefits vary across transfer directions. The paper also contains analyses of the information boundary of generated text, target risk under text-construction errors, branch agreement, and conditional refinement.

Critical strengths

For me, the paper contains novelty and strengths that can be summarized as follows:

  1. The paper possesses an interesting integration of text representations into regression adaptation, exploring how numerical labels and pseudo-labels can be re-encoded and used alongside visual evidence. This combination with auxiliary regression supervision and iterative target updates provides a coherent approach to unsupervised regression adaptation.
  2. The theoretical analysis clarifies the role and limitations of generated text, which establishes that deterministic text re-encoding does not add information conditional on the complete visual input. Its potential value is instead attributed to the representation available to a restricted regressor. Thus, these distinctions help explain both the motivation and the limitations of the method.
  3. Apart from benchmarking against baselines in three datasets, the paper also presents numerous ablation studies that verify the effectiveness of each component and address several questions about the component choices.
Critical weaknesses

However, the paper still has some weaknesses that could be addressed to further improve its positioning:

  1. My main concern is the initialization of the prototype bank and the iterative rebuilding of the target dataset. At first, the prototype bank is initialized as KK source text prototypes, where the meaning of KK is vague and requires more clarification. As the model updates and iterates, the prototype bank is not updated, while the pseudo-labels are repeatedly updated and re-encoded. Thus, my concern is (1) whether this pseudo-label updating strategy improves performance over keeping the initialization fixed, and (2) whether updating the prototype bank would bring the same effect instead of rebuilding the target domain dataset. Also, comparisons across refinement-cycle counts, together with pseudo-label error trajectories, would provide direct evidence for this central learning procedure.
  2. Some important settings require further specification, such as prototype construction (as mentioned above) and refinement settings. Although the paper mentions the bank as source text prototypes and explains template averaging, it does not clearly state how the numerical prototype vectors are selected, how many are used, or how they cover the regression space. Numerical formatting and the number of refinement cycles also need clarification.
  3. A qualitative analysis would further improve the understanding of the method with initially selected and excluded samples and recovery rate, as the behavior of difficult target samples remains insufficiently examined. Initial cosine-similarity selection retains a relative proportion of target samples, without establishing their numerical prediction accuracy, while subsequent cycles include all updated target samples, which may introduce unreliable supervision when the domain gap is large. Since the current experiments do not show when initially incorrect estimates recover or remain reinforced, I suggest that adding qualitative analysis would help identify the reliability of TCC in real-world scenarios.

Minor fix that would improve the writing clarity:

  1. In the section from lines 237-242, there is inconsistent use of the index notation ii.
Questions for authors

N/A

Reviewer_h6o5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors study unsupervised domain adaptation for image regression. Their approach converts labels into text features via a pretrained text encoder, fuses them with image features for prediction, and then uses the predictions to iteratively update the pseudo-labels and text in the target domain. The paper also includes a theoretical analysis, and reports lower average MAE than existing domain adaptation methods on dSprites, MPI3D, and BIWI. However, the writing has substantial room for improvement.

Critical strengths

The paper contains sufficient details.

The paper includes both theory and experiments.

Critical weaknesses

The writing has serious problems.

The problem setting is scattered across Section 3.1 and the beginning of Section 4 rather than stated in a dedicated problem formulation, which makes the subsequent derivations and experiments harder to follow.

L108-L124 introduce many symbols in a short space with little intuitive explanation, so it takes effort to see how this setup connects to the method that follows.

The statement in L224-225 is unclear to me: “the analysis neither uniquely determines the architecture nor guarantees the benefit of the additional RI regularizer.” If the analysis neither determines the architecture nor guarantees the benefit of RI, it is not clear what role Section 3 actually plays in motivating the proposed method.

The acronym TCC is never explicitly expanded in the paper; Sec. 1 only names the roles "competition" and "cooperation" (L61-62).

Many text-assisted vision methods are not cited. The discussion in Sec. 2.2 is very limited.

Questions for authors

See the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.