acceptodds
Under review as a conference paper at ICLR 2027

In-Context Semi-Supervised Learning via Discrete Diffusion

Abstract

In-context learning (ICL), which enables a frozen model to adapt to new tasks using labeled demonstrations, has recently attracted considerable attention. However, in many practical settings, labeled demonstrations are scarce and costly to obtain, whereas unlabeled data are abundant. This motivates in-context semi-supervised learning (IC-SSL), where a model leverages both labeled and unlabeled examples together within the context. We propose Diffusion In-Context Learner (DICL), which treats the unknown labels in IC-SSL as discrete latent variables and learns a conditional diffusion process over them. Specifically, at inference time, a bidirectional Transformer iteratively denoises missing labels while keeping the input features and observed labels fixed. During this process, DICL progressively commits its most confident predictions, allowing previously inferred labels to serve as pseudo-labeled context for subsequent predictions. Moreover, the denoising budget provides a natural inference-time compute knob, allowing a single frozen model to trade additional computation for progressively refined predictions. Experiments on in-context classification of Gaussian mixtures and in-context wireless symbol detection consistently demonstrate the effectiveness of DICL.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Discrete Diffusion Model, In-Context Learning, Semi-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9HMhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Diffusion In-Context Learner (DICL) for in-context semi-supervised learning (IC-SSL), leveraging a small set of labeled examples and a larger set of unlabeled examples. DICL uses a bidirectional Transformer trained by randomly masking a subset of unknown labels and minimizing the cross-entropy loss on masked positions. During inference, it progressively commits high-confidence pseudo-labels, incorporates them into the context for subsequent predictions, and allows the inference budget to be controlled through the number of denoising steps. The authors evaluate DICL on four synthetic settings: GMM, SubGMM, MIMO 2×2, and MIMO 4×4, comparing it with ICL, DEFINED, OICSSL, and classical statistical methods. The results show that DICL achieves lower prediction error in most settings. However, the independent contribution of the discrete diffusion objective and the effectiveness of the method on real-world data remain insufficiently validated.

Critical strengths
  1. A simple iterative inference mechanism. DICL progressively commits high-confidence labels using a single frozen model, without updating model parameters during inference. The number of inference steps also provides a controllable computational budget.

  2. Evaluation across multiple controlled settings. The authors evaluate DICL on GMM, SubGMM, and two MIMO detection settings, comparing it with several in-context learning approaches and classical statistical methods. The results demonstrate the potential benefits of exploiting unlabeled examples.

Critical weaknesses
  1. The independent contribution of discrete diffusion remains unclear, and a key baseline is missing. DICL is trained using random label masking and cross-entropy reconstruction, while its inference procedure relies on confidence-based ordering and iterative pseudo-label feedback. The current experiments do not distinguish whether the performance gains arise from the diffusion-based training objective or from the combination of masked prediction and iterative self-training. In particular, the paper lacks a direct comparison against an ordinary bidirectional prediction model using the same confidence-based ordering and multi-step pseudo-label feedback. Consequently, the core methodological contribution has not been sufficiently established.

  2. The experimental scope and robustness to pseudo-label errors are insufficiently validated. The experiments are limited to GMM, SubGMM, and simulated MIMO tasks, which may not adequately support the broader claims regarding a general IC-SSL framework. Furthermore, all visible labels during training are ground-truth labels, whereas labels progressively introduced during inference may contain errors. The paper does not sufficiently investigate how performance changes under different pseudo-label error rates or whether errors propagate across iterations and degrade subsequent predictions.

Questions for authors
  1. Could the authors include a baseline that combines an ordinary bidirectional prediction model with confidence-based ordering and multi-step pseudo-label feedback, while controlling for model architecture, training data, and inference budget as closely as possible? How large is the performance gap between this baseline and DICL?

  2. Could the authors systematically vary the error rate among visible pseudo-labels and report the prediction error at different inference steps? This would help establish DICL’s sensitivity to incorrect pseudo-labels and determine whether errors accumulate during iterative inference.

  3. Could the authors validate DICL on at least one real-world dataset, or more explicitly narrow the claims regarding a general IC-SSL framework to match the scope of the current synthetic experiments?

3 comments

W1 & Q1. Independent contribution of the diffusion objective

Thank you for your question. We ran Algorithm 1 (R=0R=0, same schedule, same budget T⋆T^\star) on the OICSSL checkpoint. OICSSL has the same architecture, training contexts, optimizer grid and selection rule as DICL and differs only in its objective.

Table R1. Long-block error (%), geometric mean over nu≥8n_u\ge8.

SettingOICSSL, 1 passrequested baseline: OICSSL + Alg. 1, N=T⋆N{=}T^\starDICL, 1 passDICL, N=T⋆N{=}T^\starbest budget: OICSSL+Alg.1 / DICL
GMM (T⋆=8T^\star{=}8)9.389.65 (+2.9%)9.559.23 (−3.4%)9.38 (N=1N{=}1) / 9.23
SubGMM (T⋆=8T^\star{=}8)2.232.49 (+11.7%)2.422.05 (−15.1%)2.23 (N=1N{=}1) / 2.05
MIMO 2×2 (T⋆=4T^\star{=}4)4.804.64 (−3.3%)5.644.55 (−19.4%)4.64 / 4.48
MIMO 4×4 (T⋆=4T^\star{=}4)18.2315.60 (−14.4%)20.3115.39 (−24.2%)15.26 / 14.86

What this shows:

  1. The objective of DICL alone does not explain the gain. In one pass, DICL is on par with OICSSL.
  2. Multi-pass decoding produces the gain, and the objective of DICL is what makes it reliable. For the DICL model, multi-pass decoding improves performance, whereas for the OICSSL model, multi-pass decoding does not necessarily lead to improvements and may even degrade performance. This behavior is expected because the training loss of DICL (Eq. (1) in the original manuscript), which samples the masking count m∼U1,…,num\sim\mathcal U\\{1,\dots,n_u\\}, encourages the model to learn to produce correct labels across all masking-count states, thereby enabling it to effectively adapt to the multi-pass decoding procedure in Algorithm 1. In contrast, the training objective of OICSSL does not encourage the model to acquire this capability, which explains why multi-pass decoding may not consistently improve its performance.

W2 & Q2. Robustness to pseudo-label errors and error propagation

Thank you for your question. We agree that all visible labels in training are ground truth, whereas pseudo-labels at inference can be wrong. Actually, we have already evaluated the performance of DICL under different inference error rates in the original paper (Tables 4 and 5). Specifically, in our wireless communication experiments, varying the signal-to-noise ratio (SNR) from 5 to 25 dB effectively corresponds to evaluating the model under different inference error rates. As shown in Tables 4 and 5, DICL achieves the best performance across most of the considered SNR levels (and thus error rates), demonstrating its robustness to incorrect pseudo-labels. Furthermore, we conduct following additional experiments to explicitly investigate the model's sensitivity to incorrect commitments.

(a) Systematic sweep of the error rate of visible pseudo-labels. We reveal half of the query labels with error rate ε\varepsilon for testing. Wrong labels are drawn either uniformly or as the model's own runner-up class, which mimics its real mistakes. We report the error on the remaining queries after one pass (DICL):

ε\varepsilon of revealed pseudo-labelsnone revealed0%2%5%10%20%30%50%
SubGMM (8,32), uniform / runner-up3.711.932.09 / 2.162.34 / 2.542.83 / 3.234.10 / 5.045.96 / 7.5312.2 / 15.1
GMM (16,32), uniform / runner-up9.443.623.86 / 4.014.20 / 4.594.82 / 5.576.38 / 7.658.06 / 9.8112.3 / 14.7

We can notice that revealed pseudo-labels keep helping (error below the "none revealed" column) up to an error rate of 10–20% on SubGMM and 20–50% on GMM. The degradation is smooth in ε\varepsilon, with no sudden collapse.

(b) Prediction error at each inference step. Below is the error of the labels committed at each step of Algorithm 1 (SubGMM (8,32), N=8N=8), against the one-shot error of the same positions:

setting (cell), NNstep 12345678
SubGMM (8,32), N=8N=8committed labels0.080.190.350.731.272.254.4812.29
same positions, one-shot0.080.190.400.901.683.276.5916.54
GMM (16,32), N=8N=8committed labels0.050.130.380.912.547.5420.0139.77
same positions, one-shot0.050.130.380.962.768.2821.4241.36

We can notice that confidence ordering commits nearly error-free labels first, below the break-even rates in (a). Labels committed later are more accurate than the one-shot predictions at the same positions, so conditioning on earlier commitments helps.

W2 & Q3. Real-world data

Thank you for your suggestion. We run in-context SSL experiments on frozen image embeddings with real-world data: We encode the 60k CIFAR-100 images with a frozen DINOv2-S/14 (CLS output), L2-normalize the embeddings and PCA-whiten them to D=20D=20 on the training classes. The 100 classes are split at random into 64 training, 16 validation and 20 test classes, so every test task uses classes that no network has seen. A task draws C=5C=5 classes, a class prior π∼Dir(2⋅15)\pi\sim\mathrm{Dir}(2\cdot\mathbf 1_5) and a random assignment of label names. The first five examples cover the classes and all others are i.i.d. from π\pi, drawn without replacement; nl∈5,10n_l\in\\{5,10\\} and nu∈1,…,64n_u\in\\{1,\dots,64\\}. All learned methods use the 4-layer Transformers and are trained with 5×1055\times10^5 training tasks. Each training context receives a random orthogonal map of the features and a random permutation of the label names.

n_ln_uLogRegQDAEM-K=4LabelPropICLDEFINEDOICSSLDICL-N=1DICL-N=2DICL-N=4
51625.4125.6522.4123.5924.3124.1820.2520.3820.2320.25
53225.4425.6920.8622.6924.3723.8118.3618.6218.2118.23
56425.4525.6919.5021.9924.3723.2016.5817.0716.4416.41
101618.3517.3715.5917.8016.7116.3814.3214.2114.2214.26
103218.3417.3914.5817.1816.7416.0713.2313.1313.0613.06
106418.3217.3413.5016.6316.7015.5812.1812.1112.0011.98

We can observe that DICL outperform all classical methods (including semi-supervised EM and label propagation tuned on the validation set), as well as labeled-only ICL, DEFINED and OICSSL, on real embeddings of unseen classes, demonstrating the effectiveness of DICL on tasks with real-world data.

We hope that our response has addressed your concerns. Please let us know if you have any further questions.

Dear Reviewer 9HMh,

Thank you again for your time and valuable feedback.

We hope our rebuttal has adequately addressed your concerns. As the rebuttal period is approaching its deadline (November 18, Anywhere on Earth), we would greatly appreciate it if you could let us know whether you have any remaining questions or concerns. We would be happy to provide further clarification.

Reviewer_WS3gOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates missing labels in in-context semi-supervised learning as discrete latent variables. DICL trains a bidirectional Transformer with random label masking and predicts missing labels through repeated confidence-ordered commitment. The same frozen model can use different numbers of denoising passes. Experiments cover Gaussian-mixture classification and MIMO symbol detection, with comparisons to one-shot IC-SSL, decision-feedback ICL, and tuned statistical baselines.

Critical strengths
  1. The formulation is simple and provides a clear inference-compute control without retraining for each budget. Results across Gaussian-mixture and MIMO settings support the usefulness of iterative label inference.
  2. DICL and the one-shot baseline share architecture and training grids. The N=1 control, validation-selected stopping budget, remasking study, runtime measurements, and tuned statistical baselines make the comparisons informative.
Critical weaknesses
  1. The gain is not sufficiently separated from generic iterative pseudo-labeling at matched compute. The N=1 control demonstrates the benefit of additional passes, while Table 9 reports an approximately fourfold runtime increase. Comparing confidence ordering with random reveal order and a simple iterative baseline at the same budget would clarify the specific contribution of the proposed formulation. (pp. 2–4, 8, 16)
  2. The statement that training states are exactly those reachable by the sampler is too strong. Training uses correct visible labels and uniformly selected masks; inference uses potentially incorrect predictions and confidence-selected masks. Matching mask counts does not match these conditional input distributions. Please qualify the statement and assess sensitivity to incorrect committed labels. (p. 4)
Questions for authors
  1. How do confidence ordering and random reveal order compare at the same number of passes?
  2. At matched inference compute, how does DICL compare with simple iterative pseudo-labeling?
  3. How sensitive is DICL to incorrect committed labels, and what narrower training-state equivalence is actually guaranteed?
2 comments

W1, Q1, Q2. Comparing DICL with random reveal order and iterative pseudo-labeling at matched compute

Thanks for your question. We ran the following comparisons at the same number of forward passes NN, with the same architecture, training data.

Table R2. Error (%) at N=T⋆N=T^\star, geometric mean over the long-block cells (nu≥8n_u\ge8).

Same DICL weights, different reveal orderGMM (N=8N{=}8)SubGMM (N=8N{=}8)MIMO 2×2 (N=4N{=}4)MIMO 4×4 (N=4N{=}4)
confidence order (ours)9.212.044.5115.31
random order9.402.245.2017.85
iterative (left-to-right order)9.402.235.1717.90
one pass (N=1N{=}1)9.522.415.6120.19

We can observe that, with the same DICL model weights and the same inference compute NN, confidence-based ordering consistently achieves the largest performance gains across all inference budgets, outperforming both random reveal and iterative pseudo-labeling.

W2, Q3. What training-state equivalence is guaranteed, and sensitivity to wrong commitments

Thank you for pointing out that our original statement was too strong. We have revised the corresponding statement as follows: “Sampling the count m∼U1,…,num\sim\mathcal U\\{1,\dots,n_u\\} enables the model to learn to predict labels across all possible masking-count states.”

Indeed, having the same masking counts during training and inference does not necessarily imply that their conditional input distributions are identical, as the visible labels during training are ground-truth labels, whereas the pseudo-labels used during inference may be incorrect. Actually, we have already evaluated the performance of DICL under different inference error rates in the original manuscript (Tables 4 and 5). Specifically, in our wireless communication experiments, varying the signal-to-noise ratio (SNR) from 5 to 25 dB effectively corresponds to evaluating the model under different inference error rates. As shown in Tables 4 and 5, DICL achieves the best performance across most of the considered SNR levels (and thus error rates), demonstrating its robustness to incorrect pseudo-labels.

Furthermore, we conduct following additional experiments to explicitly investigate the model's sensitivity to incorrect commitments.

(a) Systematic sweep of the error rate of visible pseudo-labels. We reveal half of the query labels with error rate ε\varepsilon for testing. Wrong labels are drawn either uniformly or as the model's own runner-up class, which mimics its real mistakes. We report the error on the remaining queries after one pass (DICL):

ε\varepsilon of revealed pseudo-labelsnone revealed0%2%5%10%20%30%50%
SubGMM (8,32), uniform / runner-up3.711.932.09 / 2.162.34 / 2.542.83 / 3.234.10 / 5.045.96 / 7.5312.2 / 15.1
GMM (16,32), uniform / runner-up9.443.623.86 / 4.014.20 / 4.594.82 / 5.576.38 / 7.658.06 / 9.8112.3 / 14.7

We can notice that revealed pseudo-labels keep helping (error below the "none revealed" column) up to an error rate of 10–20% on SubGMM and 20–50% on GMM. The degradation is smooth in ε\varepsilon, with no sudden collapse.

(b) Prediction error at each inference step. Below is the error of the labels committed at each step of Algorithm 1 (SubGMM (8,32), N=8N=8), against the one-shot error of the same positions:

setting (cell), NNstep 12345678
SubGMM (8,32), N=8N=8committed labels0.080.190.350.731.272.254.4812.29
same positions, one-shot0.080.190.400.901.683.276.5916.54
GMM (16,32), N=8N=8committed labels0.050.130.380.912.547.5420.0139.77
same positions, one-shot0.050.130.380.962.768.2821.4241.36

We can observe that confidence ordering commits nearly error-free labels first, below the break-even rates in (a). Labels committed later are more accurate than the one-shot predictions at the same positions, so conditioning on earlier commitments helps.

Additionally, we evaluated the performance of DICL in ICSSL experiments using frozen image embeddings, further demonstrating its effectiveness on real-world data. Detailed results and descriptions can be found in our responses to the other two reviewers.

We hope that our response has addressed your concerns. Please let us know if you have any further questions.

Dear Reviewer WS3g,

Thank you again for your time and valuable feedback.

We hope our rebuttal has adequately addressed your concerns. As the rebuttal period is approaching its deadline (November 18, Anywhere on Earth), we would greatly appreciate it if you could let us know whether you have any remaining questions or concerns. We would be happy to provide further clarification.

Reviewer_aNBcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies in-context semi-supervised learning (IC-SSL), where a model must predict missing query labels by jointly leveraging a small set of labeled examples and a block of unlabeled queries within the same context. The authors propose Diffusion In-Context Learning (DICL), which treats missing query labels as discrete latent variables: a bidirectional Transformer is trained to recover randomly masked subsets of query labels, and inference starts from a fully masked query block and iteratively commits the most confident pseudo-labels across multiple forward passes of the same frozen model.

Critical strengths
  1. The benefit of iterative label unmasking is verified at fixed model weights against tuned classical estimators on the same grids. OICSSL and DICL share the network architecture, training data, and evaluation grids. Holding DICL's weights fixed and increasing inference passes from N=1N=1 to T⋆T^\star substantially lowers error on long unlabeled blocks—for example, from 5.561% to 3.871% on MIMO 4×44\times 4 at (4,32)(4,32), and from 2.863% to 1.855% on SubGMM at (8,64)(8,64). DICL achieves the lowest error on every SubGMM cell. In pilot-starved MIMO 4×44\times 4 (nl=2,nu=64n_l=2,n_u=64), DICL falls to 47.30% as the payload grows, whereas tuned EM-ML⋆^\star remains at 84.29%.

  2. The regimes where the method does not help are reported transparently, making its operating boundary clear. With only one or two unlabeled examples, DEFINED or EM-ML is often better (for instance, 14.93% for EM-ML versus 33.86% for DICL on MIMO 4×44\times 4 at nl=4,nu=1n_l=4,n_u=1). On standard GMM, when a class never appears in the labeled set, all in-context learners sit near 25%–27% at nl=8,nu=1n_l=8,n_u=1 (near the 27.7% missing-class rate), while logistic regression, QDA, and EM are around 30%–31%. Remasking changes geometric-mean error only by a factor of 0.975 to 1.029. These negative and boundary results clarify that the gain comes from long unlabeled blocks.

Critical weaknesses
  1. Although framed as “discrete diffusion,” the main method relies on deterministic confidence-ordered decoding, and its training objective is not decoupled from the decoding schedule. The main experiments fix the remasking fraction to R=0R=0, so Algorithm 1 deterministically commits the most confident pseudo-labels at each step—equivalent to MaskGIT/CMLM confidence-ordered decoding rather than a stochastic reverse diffusion process—and enabling remasking in Appendix D.5 changes geometric-mean error only by a factor of 0.975 to 1.029. Moreover, moving from one-shot OICSSL to DICL changes both training (random mask counts m∼U{1,…,nu}m\sim U\{1,\ldots,n_u\} instead of m=num=n_u only) and inference (NN confidence-ordered passes instead of one pass). Because the paper only varies NN on DICL's own weights and never runs Algorithm 1's multi-pass schedule on the OICSSL checkpoint, it remains unclear how much of the gain comes from random-mask training versus the multi-pass heuristic itself.

  2. All experiments are restricted to idealized synthetic generative models, and the roughly 4×4\times inference overhead over one-shot OICSSL yields mostly small gains outside an unusable high-error regime. Every benchmark (GMM, SubGMM, and simulated Rayleigh MIMO 2×22\times 2 and 4×44\times 4) has a known analytical likelihood and a matched EM estimator, with no evaluation on real tabular/classification datasets or measured channels. At the reference cell (4,32)(4,32), DICL (N=T⋆N=T^\star) costs about 3.9–4.0×3.9\text{--}4.0\times the wall-clock latency of one-shot OICSSL (Table 9), yet the accuracy gap over OICSSL is often modest (e.g., 1.855% versus 2.036% on SubGMM at nl=8,nu=64n_l=8,n_u=64, and 47.30% versus 49.65% for OICSSL on MIMO 4×44\times 4 at nl=2,nu=64n_l=2,n_u=64), with OICSSL slightly ahead on several long GMM and MIMO 2×22\times 2 blocks. In the highlighted pilot-starved MIMO 4×44\times 4 regime (nl=2n_l=2 for 4 transmit antennas), where DICL beats classical EM-ML⋆^\star (47.30% versus 84.29%), a ~47% symbol error rate remains far too high for a practical receiver and is only 2.35 percentage points below one-shot OICSSL (49.65%).

Questions for authors

Questions whose answers could change the assessment:

  1. Decoupling the training objective from multi-pass confidence decoding: If Algorithm 1's multi-pass confidence-ordered commitment schedule (R=0R=0) is applied directly to the OICSSL checkpoint (trained only on fully masked contexts m=num=n_u), what are the error rates on long unlabeled blocks across the four tasks, and how close are they to DICL (N=T⋆N=T^\star)? This directly tests whether random-count mask training contributes beyond the multi-pass decoding heuristic.
  2. Accuracy–latency trade-off against one-shot OICSSL in practical error regimes: In the highlighted pilot-starved MIMO 4×44\times 4 setting (nl=2,nu=64n_l=2,n_u=64), one-shot OICSSL already lowers symbol error from 84.29% (EM-ML⋆^\star) to 49.65%, whereas 4-pass DICL spends ~4×4\times inference latency (Table 9) to reach 47.30%—still an unusable error rate for a communication receiver. In practical low-error regimes (<5%<5\% symbol error, e.g., nl=4n_l=4), under what conditions does the accuracy gain of multi-pass DICL over one-shot OICSSL justify the 4×4\times latency overhead?
  3. Behavior beyond idealized Gaussian mixture models: All four tasks are parametric Gaussian or complex-Gaussian mixtures. Do the authors have preliminary results on non-Gaussian synthetic distributions, distribution mismatch, or a real semi-supervised classification task (e.g., in-context classification over pretrained embeddings) showing that multi-pass confidence commitment still helps without a clean analytical generative model?

Future work, not requested as a condition for changing this rating:

  • Evaluate semi-blind detection on measured wireless channels.
  • Compare at a matched compute budget with chain-of-thought EM trained as in Liu and Yang (2026).
3 comments

W1 & Q1. Decoupling the training objective from multi-pass confidence decoding

Thank you for your question. We ran Algorithm 1 (R=0R=0, same schedule, same budget T⋆T^\star) on the OICSSL checkpoint. OICSSL has the same architecture, training contexts, optimizer grid and selection rule as DICL and differs only in its objective.

Table R1. Long-block error (%), geometric mean over nu≥8n_u\ge8.

SettingOICSSL, 1 passOICSSL + Alg. 1, N=T⋆N{=}T^\starDICL, 1 passDICL, N=T⋆N{=}T^\starbest budget: OICSSL+Alg.1 / DICL
GMM (T⋆=8T^\star{=}8)9.389.65 (+2.9%)9.559.23 (−3.4%)9.38 (N=1N{=}1) / 9.23
SubGMM (T⋆=8T^\star{=}8)2.232.49 (+11.7%)2.422.05 (−15.1%)2.23 (N=1N{=}1) / 2.05
MIMO 2×2 (T⋆=4T^\star{=}4)4.804.64 (−3.3%)5.644.55 (−19.4%)4.64 / 4.48
MIMO 4×4 (T⋆=4T^\star{=}4)18.2315.60 (−14.4%)20.3115.39 (−24.2%)15.26 / 14.86

What this shows:

  1. The objective of DICL alone does not explain the gain. In one pass, DICL is on par with OICSSL.
  2. Multi-pass decoding produces the gain, and the objective of DICL is what makes it reliable. For the DICL model, multi-pass decoding improves performance, whereas for the OICSSL model, multi-pass decoding does not necessarily lead to improvements and may even degrade performance. This behavior is expected because the training loss of DICL (Eq. (1) in the paper), samples the masking count m∼U1,…,num\sim\mathcal U\\{1,\dots,n_u\\}, encouraging the model to learn to produce correct labels across all masking-count states, thereby enabling it to effectively adapt to the multi-pass decoding procedure in Algorithm 1. In contrast, the training objective of OICSSL does not encourage the model to acquire this capability, which explains why multi-pass decoding may not consistently improve its performance.

W2 & Q2. Accuracy–latency trade-off in practical error regimes

Thank you for your question. In the more practical low-error regimes regime nl=4n_l=4, we have:

settingnun_u=48163264
MIMO 4×4, nln_l=4,OICSSL18.4510.186.044.484.02
DICL NN=1 / 2 / 416.37 / 12.40 / 10.6210.07 / 6.97 / 6.026.80 / 4.73 / 4.285.49 / 4.12 / 3.865.08 / 3.96 / 3.85
N=4N=4 vs. OICSSL−42%−41%−29%−14%−4%
N=2N=2 vs. OICSSL−33%−32%−22%−8%−1%
MIMO 2×2, nln_l=4,OICSSL3.262.582.061.801.64
DICL NN=1 / 2 / 43.06 / 2.78 / 2.652.70 / 2.30 / 2.192.39 / 1.98 / 1.842.16 / 1.77 / 1.662.01 / 1.61 / 1.49
N=4N=4 vs. OICSSL−19%−15%−11%−8%−9%
N=2N=2 vs. OICSSL−15%−11%−4%−2%−2%

We can observe that, with nl=4n_l = 4 and N=T⋆=4N = T^\star = 4, DICL achieves significantly better performance than OICSSL. Furthermore, N=2N = 2, which corresponds to only twice the inference latency of OICSSL, already recovers 68–91% of the detection performance gain achieved with N=T⋆=4N = T^\star = 4. Moreover, the inference budget is a tunable parameter in DICL. One key advantage of DICL over OICSSL is its flexibility to adjust NN at inference time, enabling an accuracy–latency trade-off. This allows DICL to select an appropriate value of NN for each block according to the available latency budget.

W2 & Q3. Beyond idealized Gaussian mixtures

Thank you for your suggestion. We run in-context SSL experiments on frozen image embeddings with real-world data: We encode the 60k CIFAR-100 images with a frozen DINOv2-S/14 (CLS output), L2-normalize the embeddings and PCA-whiten them to D=20D=20 on the training classes. The 100 classes are split at random into 64 training, 16 validation and 20 test classes, so every test task uses classes that no network has seen. A task draws C=5C=5 classes, a class prior π∼Dir(2⋅15)\pi\sim\mathrm{Dir}(2\cdot\mathbf 1_5) and a random assignment of label names. The first five examples cover the classes and all others are i.i.d.\ from π\pi, drawn without replacement; nl∈{5,10}n_l\in\{5,10\} and nu∈{1,…,64}n_u\in\{1,\dots,64\}. All learned methods use the 4-layer Transformers and are trained with 5×1055\times10^5 training tasks. Each training context receives a random orthogonal map of the features and a random permutation of the label names.

n_ln_uLogRegQDAEM-K=4LabelPropICLDEFINEDOICSSLDICL-N=1DICL-N=2DICL-N=4
51625.4125.6522.4123.5924.3124.1820.2520.3820.2320.25
53225.4425.6920.8622.6924.3723.8118.3618.6218.2118.23
56425.4525.6919.5021.9924.3723.2016.5817.0716.4416.41
101618.3517.3715.5917.8016.7116.3814.3214.2114.2214.26
103218.3417.3914.5817.1816.7416.0713.2313.1313.0613.06
106418.3217.3413.5016.6316.7015.5812.1812.1112.0011.98

We can observe that DICL outperform all classical methods (including semi-supervised EM and label propagation tuned on the validation set), as well as labeled-only ICL, DEFINED and OICSSL, on real embeddings of unseen classes, demonstrating the effectiveness of DICL on tasks with real-world data.

Future-work suggestions: Evaluate semi-blind detection on measured wireless channels.

Thank you for your suggestion. In fact, we have already included a comparison with the semi-blind detection method (EM-ML, a method based on the expectation-maximization (EM) algorithm) over measured wireless channels in the experiments presented in the original paper.

Future-work suggestions: Compare at a matched compute budget with chain-of-thought EM trained as in Liu and Yang (2026).

Thank you for your suggestion. In fact, we have already cited and discussed the work of Liu and Yang (2026) in the “In-context Learning with Unlabeled Data” subsection of the Related Work section in the original paper and explained why we did not directly compare our method with theirs. Specifically, the work of Liu and Yang (2026) is primarily theoretical, and their training loss requires the model's output at each chain-of-thought (CoT) step to emulate the corresponding iteration of the expectation-maximization (EM) algorithm. Therefore, if their model is sufficiently well trained, performing tt CoT steps is essentially equivalent to executing tt iterations of the EM algorithm. For this reason, we directly compared our method with the EM algorithm in our experiments rather than with the method proposed by Liu and Yang (2026).

Dear Reviewer aNBc,

Thank you again for your time and valuable feedback.

We hope our rebuttal has adequately addressed your concerns. As the rebuttal period is approaching its deadline (November 18, Anywhere on Earth), we would greatly appreciate it if you could let us know whether you have any remaining questions or concerns. We would be happy to provide further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.