acceptodds
Withdrawn from ICLR 2027

PCoRT: Progressive Chain-of-Thought Retrieval Tuning for Robust Semi-Supervised Vision

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Semi-supervised learning (SSL) typically exploits unlabeled data through pseudo-labeling, but predictions on external or heterogeneous data can be unreliable when their distributions differ from that of the labeled target data. Recent work has introduced Chain-of-Thought (CoT) reasoning to provide additional semantic information for unlabeled visual data. However, existing approaches generally treat a CoT as a single semantic unit, leaving the distinct information conveyed by its intermediate stages underused. We propose Progressive Chain-of-Thought Retrieval Tuning (PCoRT), which preserves the stage-wise structure of CoT and progressively retrieves and fuses stage-specific semantic evidence into visual representations. PCoRT supports two settings with different uses of unlabeled data: in the in-distribution setting, CoT-enhanced predictions refine conventional pseudo-supervision; in the out-of-distribution setting, external unlabeled data provide auxiliary semantic evidence rather than direct pseudo-labels for the target task. We further distill the CoT-enhanced teacher into a visual-only student, which does not require the auxiliary CoT, retrieval, or fusion modules at deployment. Experiments on image classification, object detection, and semantic segmentation across four unlabeled-data settings show that PCoRT achieves competitive performance in all settings, with particularly strong gains in cross-dataset and heterogeneous multi-source settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Semi-supervised learning, Retrieval-Augmented Visual Tuning,Chain-of-Thought

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ePb5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PCoRT that uses four-stage descriptions generated by a vision-language model to build semantic memories. Progressive retrieval and fusion enhance a teacher, whose predictions are distilled into a visual-only student. Experiments on classification, detection, and segmentation tasks with both in-distribution and external unlabeled data are conducted.

Critical strengths

1.The framework addresses a meaningful problem: exploiting semantic information from external unlabeled images when conventional pseudo-labeling becomes unreliable. 2.Evaluations are conducted for three visual tasks and multiple data settings, while distillation preserves visual-only inference.

Critical weaknesses
  1. The main contribution of this paper is to decompose CoT descriptions into four predefined semantic stages, and progressively fuse the retrieved evidence with visual features. However, the individual components are largely established techniques, the overall framework appears to combine these existing components in a sequential architecture rather than introduce a fundamentally new learning principle.

  2. Although the manuscript discusses CERS, which is relevant to CoT-based semantic memory construction and retrieval, the experimental evaluation does not provide a sufficiently controlled comparison with its holistic reasoning retrieval mechanism.

  3. According to the ablation studies, most performance improvement comes from captions. Existing ablations do not prove the necessity of all four stages or their order; stage-skipping and reordered controls might be needed.

  4. The several closely related studies are not adequately discussed, including CFSG-CLIP (CVPR 2025), SemiDAViL (CVPR 2025), ViLaDiff (AAAI 2026), and Thinking in Latent Space: Progressive Multimodal Simplification for Visual Reasoning (ICML 2026). These studies have investigated several concepts central to the present manuscript, including progressive semantic guidance, language-enhanced semi-supervised segmentation, and progressive multimodal reasoning with retrieved visual evidence.

Questions for authors
  1. What percentage of PCoRT's performance gain can be attributed specifically to progressive retrieval, after controlling for additional semantic knowledge supplied by Qwen2.5-VL-7B-Instruct?
  2. Have the authors evaluated reversing or randomly permuting the four stages? If not, what evidence supports the claim that their specified order reflects a necessary reasoning progression?
  3. See the Critical Weaknesses.
Reviewer_CKzROpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a semi-supervised learning framework that leverages structured COT reasoning from VLMs to improve learning from unlabeled data under distribution shifts. The method progressively incorporates stage-specific semantic information to refine teacher predictions and distills the enhanced knowledge into a student model. Experiments across classification, detection, and segmentation demonstrate promising performance gains.

Critical strengths
  1. The motivation is strong, OOD is a very important problem in the semi-supervised learning.
  2. The method is evaluated across classification, detection, and segmentation under four different settings, demonstrating its applicability to multiple vision tasks.
Critical weaknesses
  1. The novelty of this work is limited. This paper uses Qwen2.5 to provide additional semantic knowledge, but most baselines do not have access to such information. For a powerful VLM, nothing in the datasets used by the authors is out-of-knowledge.
  2. From the ablation studies, it seems that most of the improvement comes from introducing textual information, while the additional benefit of progressive CoT is relatively small.
  3. Some implementation details are missing or unclear, particularly the training hyperparameters etc. These details are important for reproducing the results.
Questions for authors
  1. It would be good to see what happens if the reasoning stages are shuffled.
  2. How much of the performance improvement comes from the pretrained VLM rather than the proposed progressive retrieval mechanism? Could the authors demonstrate this?
Reviewer_gXMROpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Progressive Chain-of-Thought Retrieval Tuning, a semi-supervised visual learning framework that uses a pretrained VLM to generate structured CoT descriptions of unlabeled images and progressively retrieves and fuses stage-specific semantic information (use embeddings of the descriptions). The enhanced teacher is distilled into a visual-only student for efficient inference.

Critical strengths

Exploring semantic signals from pretrained VLMs is a potentially useful direction for improving SSL beyond conventional in-distribution settings.

Critical weaknesses

Outdated and unfair experimental comparisons

The proposed method leverages semantic knowledge from a pretrained Qwen2.5-VL-7B model, yet most experimental comparisons are against conventional SSL methods developed before the widespread adoption of foundation models.

  • If Qwen2.5-VL-7B can generate reliable semantic descriptions and task-specific visual cues, why not directly use the same VLM to generate pseudo-labels for downstream training? Authors should add this baseline.
  • Since PCoRT explicitly benefits from pretrained foundation-model knowledge, authors shoudl compare it against approaches that use foundation models like papers below instead of conventional SSL methods from 2019–2024
  • Figure 3 uses a benchmark suite introduced in [2], but does not cite or discuss that work. For example, [3] uses this benchmark and setting and [3] will acknowledge and cite the related work.

[1] Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024)

[2] Revisiting Semi-Supervised Learning in the Era of Foundation Models (NeurIPS 2025)

[3] AgentSSL: Can MLE Agents Leverage Unlabeled Data? (COLM 2026)

Bad presentaion and writing

  • The paper did not provide sufficient implementation details to understand or reproduce the reported results. Examples include backbones and training details for classification, detection, and segmentation experiments. Are these settings comparable to existing baselines?
  • In section "Multimodal Fusion Strategies,", the authors compare fixed-weight, residual, gated, reliability-aware, cross-attention, but do not explain how most of these strategies are implemented.
  • Figure 6 already includes image-specific answers such as the object category ("frog"), rather than general instructions. It should be figure error.
  • Figure 2 is also excessively dense, making the proposed training and supervision pipelines difficult to follow.
Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.