acceptodds
Under review as a conference paper at ICLR 2027

ALICE: Aligning Latent Reasoning with Chain-of-Thought for Multimodal-LLM Embeddings

11 views

Abstract

Recent multimodal large language model (MLLM) embedders show potential to improve retrieval by generating explicit chain-of-thought (CoT) rationales before the final representation. However, this introduces substantial inference latency and may incur information loss due to discrete autoregressive generation. In this paper, we investigate bringing latent reasoning into multimodal embedding. We first show that latent tokens trained with curriculum-based next-token prediction tend to collapse to predicting a narrow set of CoT-initial tokens, suggesting a bias toward local token prediction over holistic reasoning semantics. Based on this observation, we propose **ALICE** (**A**ligning **L**atent Reason**i**ng with **C**hain-of-Thought **E**mbeddings), a direct *hidden-state to embedding-space* alignment approach: we insert a short block of `<latent>` tokens into the query and train their hidden states to match per-step CoT anchors from a separately trained multi-granularity reference embedder, followed by a free-form contrastive stage that adapts the latents for retrieval. ALICE thus requires a single forward pass at inference, without autoregressive generation. On MMEB, ALICE achieves strong results across the 2B–8B scale, reaching 74.8 with Qwen2-VL-7B, with minimal additional latency. Analysis further reveals that the learned latents exhibit sequential structure, encode instance-specific information, and become increasingly aligned with the final embedding, suggesting reasoning-like behavior in latent space.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
MLLM Embedding, Latent Reasoning, Feature Alignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_nSahOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ALICE, a multimodal LLM embedder that inserts 15 learnable latent tokens between the query and the embedding token. The goal is to keep the gains of chain-of-thought (CoT) embedders without generating text at inference.

Training is split into 3 stages:

  1. Qwen3-VL-235B writes a 5-step CoT for each MMEB training pair, with the target visible. A reference embedder then learns to map the full CoT and each single step to the target embedding.
  2. The latent tokens are mean-pooled into 5 groups, and each group is aligned with InfoNCE to the embedding of its step. A query-to-target contrastive loss is trained alongside.
  3. The alignment loss is dropped, and training continues with the contrastive loss alone.

The approach is motivated by an initial analysis. Latents trained with a COCONUT curriculum decode mostly to tokens that start CoT steps. MMEB scores each task by Hit@1, the share of queries whose correct target ranks first among the candidates. On MMEB, ALICE reaches 70.6 to 74.8 Hit@1 across seven Qwen-VL backbones from 2B to 8B. Analyses of a 5-token variant are considered to support reasoning in latent space.

I currently consider this paper to be on the fence, cutting right between acceptance and rejection. The novelty is generally limited in the crowded field, but the experiments do add value. However, such a work needs to pass high quality bar for the experimental setup that this work does not quite yet meet (cf. weaknesses). I invite the authors to rebut my review.

Critical strengths
  • Relevance: Embedders that generate a CoT before embedding are slow at query time. Table 2 reports 2,018 ms per query for RGE and 7,229 ms for UME-R1, vs. 6.7 ms for ALICE. ALICE gets close to or above these methods on MMEB with a single forward pass.
  • Broad baseline coverage: Results cover 7 backbones from 3 Qwen-VL generations.
  • Solid experimental ablations: Aligning the latents to CoTs from other instances scores 63.0, vs. 63.9 without alignment (Table 8). Aligning them to a generic task specification scores 64.0. Both results show that the useful signal is instance-specific.
Critical weaknesses

The authors claim that the latent tokens carry reasoning distilled from the CoT. Tables 3 and 5 do not separate the latent tokens from the teacher signal. Specifically:

  • Unaligned latents score 66.0 vs. 65.8 without latents.
  • Aligning the latents to the plain target embedding, excl. CoT, reaches 70.2. Full ALICE reaches 71.9.
  • Most of the alignment gain thus appears without CoT steps. The CoT-specific part is 1.7 points from a single run.

The missing variant distills the same reference-embedder signal into the embedding token, excl. latents, at matched epochs. An epoch-matched baseline is also missing: ALICE trains the embedder for 2 contrastive epochs, while the baseline trains for one.

Experimental setup: All results come from single runs. The paper describes no validation split & every ablation reports MMEB test scores. After the latent-budget sweep in Fig. 4, the paper sets the best-scoring budget as the default (Sec. 4.3). If design choices were made on these scores, the reported gains are overstated.

The most comparable baselines were not evaluated:

  • TTE is cited and appears in Table 7, but its MMEB-v1 results are absent from Table 1. With Qwen2-VL-7B, TTE reaches 73.3 with a small trained reasoner and 78.5 with a 72B reasoner.
  • MMEmb-R1 uses the same Qwen-VL backbones and reports 74.8 on the 36 image tasks at 4B, vs. 72.8 for ALICE.
  • Single-pass latent-token embedders such as LaME and TTE-Flash are the closest competitors to the efficiency claim. Both report MMEB-v2 results and should be compared or at least discussed.
  • Only Qwen2.5-VL-3B has an in-house baseline trained with ALICE's recipe. For the other backbones, gains over copied numbers mix the method with the recipe and the data.
Questions for authors
  1. What does a model reach that distills the reference-embedder signal into the embedding token, without latents, at equal training budget? An example is an auxiliary loss that aligns the embedding token to the CoT embedding or to the final-step embedding.
  2. Which training data produced each ALICE row in Table 1? The Qwen2-VL-2B and Qwen2-VL-7B rows match the image results of the MMEB-v2 models in Tables 7 and 11 in every column, while the other backbones differ. Sec. 4.1 states that training uses the MMEB training split. If these two rows come from the 1.8M-example model, what do image-only models reach?
  3. How close is each latent group to its step anchor after the free-form stage? It would be interesting to assess the cosine similarity, or the accuracy of retrieving the correct step among the steps of the same instance.
  4. How were the number of latents, the objective, and the alignment target chosen? Did you use data held out from the MMEB test sets? Moreover, results of Tables 3 and 5 and for Fig. 4 would be ideally supported by 3 seeds.
  5. Does ALICE help on reasoning-intensive retrieval benchmarks such as MRMR, where a reasoning benefit should be largest?
10 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer nSah (1/3)

Q1: What does a model reach that distills the reference-embedder signal into the embedding token, without latents, at equal training budget?

We thank the reviewer for this suggestion. We have added a control that uses no latent tokens and directly distills the CoT representation from the same reference embedder into the final embedding token. The experiment uses the 160K training data and Qwen2.5-VL-3B setting from Appendix C, with the same S2 training budget. All variants are evaluated after S2.

Specifically, the frozen reference embedder encodes the full CoT. A cosine loss aligns the final <emb> representation, computed without latent tokens, with the CoT representation produced by the reference embedder. We test two settings: cosine-loss distillation alone, and cosine-loss distillation combined with the query–target retrieval contrastive loss:

MethodOverallCLSVQARetrievalGroundingINDOOD
No latents, cosine distillation only50.054.944.049.654.152.946.4
No latents, cosine distillation + retrieval contrastive learning59.958.558.258.173.663.954.9
ALICE (S2)67.962.367.168.083.572.262.4

Direct CoT distillation combined with retrieval contrastive learning achieves 59.9, compared with 67.9 for ALICE. Our reference embedder is designed to convert CoT information into retrieval-relevant semantic anchors for intermediate supervision, rather than to provide a complete target representation for the final query embedding. In ALICE, these anchors guide the intermediate latent states, while the final embedding learns to integrate the query and latent information through the retrieval objective. The results support this use of CoT representations as intermediate supervision over directly distilling them into the final embedding. We have added this control and its description to Appendix C / Table 11 of the revised manuscript.

We also address the request for an epoch-matched baseline in the Weaknesses section: we train both the standard embedding baseline without latents and the baseline with the same number of latent tokens but no CoT alignment for two epochs, matching the combined training epochs of S2 and S3 in full ALICE.

MethodTraining epochsCLSVQARetrievalGroundingINDOODOverall
Standard embedding, no latents259.162.868.789.373.158.666.7
Latent-only, no CoT alignment260.662.768.190.473.159.367.0
Full ALICE266.868.772.490.677.065.471.9

After two epochs, the standard and latent-only baselines achieve overall scores of 66.7 and 67.0, respectively, compared with ALICE's 71.9. These results indicate that additional training epochs, with or without learnable latent tokens, are insufficient to explain ALICE's improvement.


Q2: Which training data produced each ALICE row in Table 1? If these two rows come from the 1.8M-example model, what do image-only models reach?

We thank the reviewer for pointing this out. The Qwen2-VL-2B and Qwen2-VL-7B rows in Table 1 mistakenly reported results from checkpoints trained on the MMEB-v2 mixture. We have replaced them with the corresponding results from models trained on the MMEB image-only data. All ALICE rows in the revised Table 1 now report results from models trained on the MMEB image-only training data. The original and corrected results are shown below:

Result versionBackboneCLSVQARetrievalGroundingINDOODOverall
Original Table 1Qwen2-VL-2B69.265.371.988.175.365.971.1
Original Table 1Qwen2-VL-7B71.870.375.890.679.668.874.8
CorrectedQwen2-VL-2B69.465.672.088.175.965.671.3
CorrectedQwen2-VL-7B72.270.675.490.780.068.474.8

The Overall score for Qwen2-VL-2B is updated from 71.1 to 71.3. For Qwen2-VL-7B, the Overall score remains 74.8 to one decimal place.

We sincerely apologize for this error and the resulting confusion. The correction affects only these two rows. The ablations analyzing the method's contributions in §4.3 and the relevant appendices all use Qwen2.5-VL-3B under controlled settings, so those ablations and their conclusions are unaffected.

AuthorsOpenReview ↗

Rebuttal to Reviewer nSah (2/3)

Q3: How close is each latent group to its step anchor after the free-form stage?

We thank the reviewer for this suggestion. We assess the correspondence between latent groups and CoT anchors through within-instance step matching. We divide the 15 latent tokens into five groups and mean-pool each group, then compute cosine similarities between each group representation and the five CoT step anchors encoded by the fixed reference embedder. Matching accuracy is the proportion of cases in which the originally corresponding step ranks first.

To further examine retention of step correspondence, we sample 200 examples from each training subset, for a total of 4,000 examples, and compare S2 and S3 on the same examples:

Sample scopeAfter S2After S3
200 samples/subset * 20 subsets59.92%54.60%

Step correspondence remains observable after S3, with a matching accuracy of 54.60%, compared with 59.92% after S2. Our free-form training is inspired by latent-reasoning methods such as COCONUT, which shift to final-task supervision in the last training stage. In ALICE, S2 provides step-level semantic supervision, while S3 removes the alignment loss and allows the intermediate representations to adapt to the retrieval objective. In the stage ablation in Table 3, adding S3 improves overall retrieval performance from 69.7 to 71.9. These results suggest that the intermediate representations remain associated with their corresponding CoT steps after adaptation to retrieval.


Q4: How were the number of latents, the objective, and the alignment target chosen? Results of Tables 3 and 5 and Fig. 4 would ideally be supported by 3 seeds.

We agree that experiments with multiple seeds help assess stability. We have completed three-seed experiments for the four ALICE latent-budget configurations with relatively close scores in Fig. 4 and added means and standard-deviation error bars to the revised figure. We also provide three-seed results for ALICE with Qwen2-VL-2B on MMEB-v2, with an overall score of 64.99±0.01 (mean ± standard deviation). The complete results are included in Appendix B.2 / Table 9 of the revised manuscript and in our response to Reviewer FSQ3, Question 5.

Computational resources and the rebuttal timeline currently prevent us from completing three training seeds for every configuration in Tables 3 and 5. For these ablations, our reporting follows the practice of related work such as VLM2Vec, UME-R1, Embed-RL, PLUME, RGE, and CAFe, whose relevant ablations report point estimates without standard deviations across training seeds. The performance gaps in the main comparisons of these tables are larger, and we expect their conclusions to be less sensitive to training randomness.


Q5: Does ALICE help on reasoning-intensive retrieval benchmarks such as MRMR?

We thank the reviewer for this suggestion. We have added an evaluation of ALICE on the reasoning-intensive retrieval benchmark MRMR, covering all 11 tasks:

MethodBackboneArtMedSciHumMathPhyEngBusNegDesignTrafficAVG(11)
OpenCLIPViT-G/14-1B56.017.933.222.05.75.07.09.713.08.112.417.3
PLUMEQwen2-VL-2B68.747.763.654.719.230.329.244.57.544.321.939.2
UME-R1Qwen2-VL-2B70.648.864.958.020.930.026.846.77.042.627.840.4
ALICEQwen2-VL-2B73.548.358.956.521.630.128.446.56.041.937.540.8
VLM2VecQwen2-VL-7B37.219.934.129.412.215.316.935.45.03.477.5319.7
UME-R1Qwen2-VL-7B77.855.772.964.127.239.232.247.87.561.941.748.0
ALICEQwen2-VL-7B75.754.368.360.131.143.134.053.09.570.040.349.0

With Qwen2-VL-2B, ALICE achieves an average score of 40.8, compared with 40.4 for UME-R1 and 39.2 for PLUME. With Qwen2-VL-7B, ALICE achieves 49.0, compared with 48.0 for UME-R1. These results are included in Appendix B.3 / Table 10 of the revised manuscript.

AuthorsOpenReview ↗

Rebuttal to Reviewer nSah (3/3)

Additional response to weaknesses: Baseline comparisons and controlled experiments

We thank the reviewer for highlighting these related methods. We have expanded the comparisons in the revised manuscript. Table 1 now includes the available MMEB-v1 results for TTE, MMEmb-R1, and LaME, while Appendix B / Table 7 includes the MMEB-v2 comparisons with MMEmb-R1, LaME, and TTE-Flash. We have also expanded the discussion in the Related Work section.

We use Qwen2.5-VL-3B as a representative backbone for systematic ablations, as its model size makes it practical to evaluate a broad range of design choices under controlled conditions. By keeping the backbone and training data fixed and matching the training budget within each comparison, we can assess the contributions of ALICE's different designs. Our conclusions about these design choices are based on these controlled experiments. We then evaluate the complete method across seven backbones to examine whether its effectiveness extends to different model sizes and generations.

Reviewer_FSQ3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper observes that explicit-CoT embedding is expensive, and that latents trained with a COCONUT-style next-token curriculum tend to predict the first token of CoT steps. It proposes ALICE: insert 15 <latent> tokens, first train a reference embedder that provides embedding anchors for the full and per-step CoT, then align the latents to these anchors with stepwise InfoNCE, and finally fine-tune freely with the contrastive loss only. Inference is a single forward pass. On MMEB, Qwen2-VL-7B reaches 74.8, and on CIRR latency is only 1.4 ms above VLM2Vec. The authors also analyze latent behavior through similarity, answer-token presence, and KV-cache interventions.

Critical strengths

Practical, solid results: Single forward pass; on CIRR latency is 6.7 ms versus 2018 ms for RGE and 7229 ms for UME-R1, with the highest Hit@1 (58.0). On MMEB it improves across 2B–8B and Qwen2/2.5/3-VL backbones. This directly eases the deployment problem of reasoning-based embedders.

Fairly systematic ablations with controlled comparisons: Table 3 separates "adding latents only" (65.8→66.0, +0.2), "CoT alignment" (paper states +3.9; in the table 66.0→69.7 is +3.7) and "free-form training" (+2.2); Tables 4/5 compare loss forms and alignment targets; appendix Table 8 adds controls such as "CoT from other instances" (63.0 < 63.9 without alignment) and "shuffled steps").

Critical weaknesses

Attribution of the gain is not clean; the benefit of "latent reasoning" may be overstated. In Table 5, aligning to the target embedding alone (no CoT) already lifts the score from 66.0 to 70.2, versus 71.9 for ALICE — stepwise CoT anchors add only about 1.7 points over target-embedding alignment. In appendix Table 8 (160K subset), direct target embedding gives 66.1, shuffled CoT steps 66.7, and original order 67.9 (no alignment: 63.9), suggesting most of the gain comes from "aligning to target-relevant semantics", with the independent contribution of step structure about 1.2–1.9 points. Moreover, the CoT is produced by a 235B teacher given the ground truth, so the data and teacher differ from those of Embed-RL/UME-R1 in the comparisons; the non-reasoning baseline (Table 3, row 1) has only one epoch of contrastive training, whereas ALICE's embedder undergoes two stages (alignment + free-form, one epoch each), so a training-budget-matched non-reasoning baseline is missing.

The motivating diagnosis and the generality claims are too strong. The "first-token overlap" (79.8% vs. 21.1%) is not linked to retrieval performance: in Fig. 4 the COCONUT-like model still beats the non-reasoning baseline (65.8) by 3–4 points. The authors themselves acknowledge in the Limitations that they cannot distinguish "integrating complementary semantic information" from "compositional reasoning", so the "reasoning-like behavior" wording should be toned down. In addition, on MMEB-V2 ALICE (Qwen3-VL-4B, 67.6) is below Embed-RL (68.1) and lags on video tasks, so the advantage does not hold across all modalities; the paper reports no multi-run variance for performance metrics (only latency has ±).

Questions for authors

1: Can you provide a non-reasoning baseline matched to ALICE's total training budget (epochs/GPU-hours), and a "learnable tokens + extra epoch" baseline? 2: Can you retrain one representative baseline (UME-R1 or Embed-RL) on the same CoT data and with the same teacher? 3: Does the "first-token overlap" correlate with MMEB scores (e.g., across curriculum settings or latent budgets)? 4: The reference embedder is trained on "CoT→target" pairs, and the CoT generator sees the ground truth. Please report the gain over the baseline on the 16 OOD datasets, as well as a version where the CoT generator does not see the target. 5: Please report multi-seed variance for the main results and analyze why ALICE lags on video/temporal tasks in MMEB-V2.

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer FSQ3 (1/3)

Q1: Can you provide a non-reasoning baseline matched to ALICE's total training budget (epochs/GPU-hours), and a "learnable tokens + extra epoch" baseline?

Table 3 shows that ALICE after S2 CoT alignment alone substantially outperforms the original baseline overall, supporting the effectiveness of this supervision. To examine whether ALICE's gains can be explained by additional training epochs, with or without learnable tokens, we extended both baselines to two training epochs, matching full ALICE.

MethodTraining epochsCLSVQARetrievalGroundingINDOODOverall
Standard embedding, no latents259.162.868.789.373.158.666.7
Latent-only, no CoT alignment260.662.768.190.473.159.367.0
Full ALICE266.868.772.490.677.065.471.9

Longer training improves both baselines, but they still fall short of ALICE, which retains an advantage across all task categories. These results show that additional training epochs and latent tokens alone are insufficient to explain ALICE's gains, and that CoT-guided alignment provides additional value, further addressing the attribution concern in W1.


Q2: Can you retrain one representative baseline (UME-R1 or Embed-RL) on the same CoT data and with the same teacher?

We thank the reviewer for this suggestion. We agree that matching training data and CoT sources is important for a fair comparison. Different methods rely on different annotation formats, such as candidate-side reasoning text in UME-R1 and explicit visual evidence in Embed-RL. Directly replacing their original data with ALICE's annotations, which contain only query-side CoTs, may fail to preserve their full design and compromise the fairness of the comparison. We therefore reverse the direction of the data-matching control: we retrain ALICE with Qwen2-VL-2B using 1.46M publicly released SFT training examples from UME-R1, with the provided query-side CoTs.

Specifically, we retain the original query–positive pairs and query CoT content, and split each CoT into five segments in its original order to match ALICE's supervision format. We use these data for all training stages: reference teacher training, S2, and S3, and evaluate on all 78 MMEB-v2 tasks. This control is now included in Appendix B.1 / Table 8 of the revision. All methods below use the Qwen2-VL-2B backbone.

ModelImage / CLSImage / QAImage / RETImage / GDImage / Avg.Video / CLSVideo / QAVideo / RETVideo / MRETVideo / Avg.VisDoc / VDRv1VisDoc / VDRv2VisDoc / VRVisDoc / OODVisDoc / Avg.All
# Datasets101012436555318104642478
UME-R1 (Table 7)64.862.867.677.266.644.351.232.939.742.272.446.279.237.263.960.1
ALICE (Table 7)69.265.371.988.071.151.648.033.234.642.778.148.684.964.272.665.0
ALICE (UME-R1 data)68.465.571.386.370.551.753.535.034.344.776.355.279.765.371.865.0

Compared with UME-R1, ALICE trained on the same SFT dataset achieves a higher overall score (65.0 vs. 60.1) and higher average scores across all three modalities. These results show that ALICE's effectiveness is not restricted to our own CoT annotations.

AuthorsOpenReview ↗

Rebuttal to Reviewer FSQ3 (2/3)

Q3: Does the "first-token overlap" correlate with MMEB scores?

We thank the reviewer for this question. We use first-token overlap to diagnose the vocabulary-level prediction tendencies of latent states; we do not directly link this metric to retrieval performance. Under autoregressive supervision, high-probability vocabulary predictions from these states concentrate on CoT step-initial tokens. Although PLUME and our reproduced COCONUT-like model demonstrate the effectiveness of autoregressive latent training for retrieval, we ask whether next-token prediction is the most suitable way to transfer CoT information into retrieval representations. We raise this question because autoregressive next-token supervision is designed to support subsequent text generation, whereas embedding-based retrieval requires a continuous representation without generating text. This difference motivates ALICE's direct alignment of intermediate latent representations with CoT semantic anchors. We evaluate the retrieval benefits of this design through the comparisons in Fig. 4 and the ablations in Tables 3 and 11 (originally Table 8).

To more accurately describe the observation, we have refined its interpretation in the introduction and revised the reasoning-like behavior wording in the abstract and conclusion, clarifying the relationship between the diagnosis of prediction tendencies, the motivation for the design, and the validation of retrieval performance.


Q4: Please report the gain over the baseline on the 16 OOD datasets, as well as a version where the CoT generator does not see the target.

We thank the reviewer for this question. We re-annotate the same 160K training subset without giving the CoT generator access to the target. Following the setting in Appendix C, we use Qwen2.5-VL-3B and evaluate directly on MMEB-v1 after one epoch of S2, without subsequent S3 training. The results are as follows:

CoT supervision settingINDOOD (16 datasets)Overall
No CoT supervision, no latent tokens68.457.863.7
No CoT supervision, 15 latent tokens68.857.663.9
CoT generated without target access71.562.267.4
CoT generated with target access (our setting)72.262.467.9

With target-conditioned CoTs, ALICE improves the average score on the 16 OOD datasets from 57.6 to 62.4 over the latent-only baseline. Without target access during CoT generation, it still reaches 62.2, retaining nearly all of this improvement. Its overall score is also close to the target-conditioned setting (67.4 vs. 67.9). These results show that the retrieval benefits of CoT supervision persist without providing the target during annotation. Our use of GT-assisted annotation and verification serves as an additional quality-control safeguard, helping keep the generated explanations consistent with the correct query–target match.

We further evaluate ALICE on MRMR [1], a reasoning-intensive retrieval benchmark, to assess its generalization beyond MMEB. With Qwen2-VL-2B, ALICE achieves 40.8, compared with 40.4 for UME-R1 and 39.2 for PLUME. With Qwen2-VL-7B, ALICE achieves 49.0, compared with 48.0 for UME-R1. The results are included in Appendix B.3 (Table 10) of the revised manuscript.

MethodBackboneArtMedSciHumMathPhyEngBusNegDesignTrafficAVG(11)
OpenCLIPViT-G/14-1B56.017.933.222.05.75.07.09.713.08.112.417.3
PLUMEQwen2-VL-2B68.747.763.654.719.230.329.244.57.544.321.939.2
UME-R1Qwen2-VL-2B70.648.864.958.020.930.026.846.77.042.627.840.4
ALICEQwen2-VL-2B73.548.358.956.521.630.128.446.56.041.937.540.8
VLM2VecQwen2-VL-7B37.219.934.129.412.215.316.935.45.03.477.5319.7
UME-R1Qwen2-VL-7B77.855.772.964.127.239.232.247.87.561.941.748.0
ALICEQwen2-VL-7B75.754.368.360.131.143.134.053.09.570.040.349.0

[1] MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval. ICLR 2026

AuthorsOpenReview ↗

Rebuttal to Reviewer FSQ3 (3/3)

Q5: Please report multi-seed variance for the main results and analyze why ALICE lags on video/temporal tasks in MMEB-V2.

We thank the reviewer for this suggestion. We have completed a three-seed evaluation of ALICE with Qwen2-VL-2B on all 78 MMEB-v2 tasks. Following the column order of Table 7, the table below reports the mean ± standard deviation across three training seeds. These results are also included in Appendix B.2 (Table 9) of the revised manuscript.

StatisticImage / CLSImage / QAImage / RETImage / GDImage / Avg.Video / CLSVideo / QAVideo / RETVideo / MRETVideo / Avg.VisDoc / VDRv1VisDoc / VDRv2VisDoc / VRVisDoc / OODVisDoc / Avg.All
# Datasets101012436555318104642478
Mean69.2265.4171.7687.5771.0551.5848.3233.4434.6742.8278.4147.7384.8164.2272.5364.99
SD0.020.110.110.450.060.161.060.230.200.340.251.470.080.050.180.01

For video performance, our main hypothesis concerns the types of supervision provided by the training data. The video portion of the current MMEB-v2 training set mainly covers QA, caption retrieval, and text-to-video retrieval. Moment retrieval additionally requires precise localization of the temporal interval relevant to a query. The current training tasks provide limited direct supervision for this temporal localization requirement, making task coverage a plausible explanation for the observed differences in performance.

The data control in our response to Q2 also provides a clue: training ALICE with Qwen2-VL-2B on UME-R1's SFT data improves the video average from 42.7 to 44.7 (Appendix B.1 / Table 8). To investigate this improvement, we compared annotations on matched video examples and found that UME-R1's complete query-side annotations are on average 2.36× as long as ours. These longer annotations may contain finer-grained video details, suggesting that annotation detail could be an additional factor affecting video performance.

Therefore, a promising future direction would be to incorporate explicit supervision for temporal localization and cross-frame event relations to help intermediate representations capture finer-grained temporal information.

Reviewer_xRZiOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ALICE, a latent reasoning framework for MLLM-based multimodal embedding tasks. Instead of generating explicit CoT sequences during inference, ALICE introduces a fixed number of latent tokens to produce retrieval representations in a single forward pass. Unlike autoregressive latent reasoning methods that rely on next-token supervision, ALICE directly aligns grouped latent hidden states with step-level CoT embeddings obtained from a separately trained reference embedder. The framework adopts a two-stage training procedure consisting of CoT-anchored latent alignment and free-form contrastive adaptation. During inference, the model processes the query together with the latent tokens and a final embedding token without autoregressive generation. Experiments on MMEB across multiple 2B–8B backbones demonstrate competitive retrieval performance, while introducing only 1.4 ms of additional latency over a standard embedder in the reported CIRR efficiency evaluation.

Critical strengths

S1. The motivation is clear and has practical significance. This paper points out a significant limitation of the embedding method based on autoregressive thought chains (CoT), namely its large inference overhead. By replacing explicit CoT generation with latent-space representations derived from a single forward pass, the paper offers a viable approach to enhancing the efficiency of multimodal retrieval.

S2. Sound methodological design. The proposed "CoT-anchored alignment" mechanism provides direct semantic supervision for latent-space representations, moving beyond a reliance solely on "next-token prediction." The two-stage training pipeline is well-structured, and ablation studies confirm the effectiveness of the key components.

S3. Comprehensive experimental evaluation. The approach is evaluated across various Multimodal Large Language Model (MLLM) backbones and a range of multimodal retrieval tasks. The results demonstrate that the method achieves competitive performance while maintaining low inference overhead.

Critical weaknesses

W1. Limited evidence for latent reasoning. While analyses of latent representations reveal meaningful sequential structure and information relevant to retrieval, it remains unclear whether these observations reflect genuine multi-step reasoning or merely the accumulation of useful semantic features.

W2. Dependence on target-conditioned CoT supervision. The CoT annotations are generated with access to ground-truth targets, and the reference embedder further aligns individual CoT steps with target representations. Although the authors introduce safeguards against explicit target leakage, it is unclear to what extent the learned latent representations capture independent reasoning processes rather than target-oriented semantic decomposition.

W3. Training overhead and generalization limitations. While inference is efficient, ALICE requires large-scale CoT annotation and additional reference model training. The overall cost of constructing these annotations is not fully discussed. Also the MMEB-v2 results indicate weaker performance on some video-related tasks, suggesting potential limitations of the fixed-length latent representation.

Questions for authors

Thank you for your work. I appreciate the clear motivation and the extensive experimental evaluation. The proposed framework demonstrates promising performance and inference efficiency. However, I have a few questions regarding the interpretation of the learned latent representations and the role of CoT supervision.

Q1: Distinguishing latent reasoning from semantic representation learning. Fig. 5 shows that latent tokens capture retrieval-relevant information and contribute to final predictions. However, these observations do not conclusively establish multi-step reasoning. How do the authors distinguish the proposed latent reasoning from progressive semantic feature aggregation, and what operational criteria define reasoning in this embedding setting?

Q2: The role of target-conditioned CoT supervision. Section 3.2 uses ground-truth targets during CoT generation, while the reference embedder aligns individual CoT steps with target embeddings. Despite safeguards against target leakage and the reported supervision ablations, how does this training procedure encourage genuine reasoning rather than target-oriented semantic decomposition? How should individual latent steps be interpreted under this supervision?

Q3: Practical trade-offs and generalization. While ALICE achieves impressive inference efficiency, it introduces additional costs from reference model training and large-scale CoT annotation. Could the authors clarify the overall cost, including CoT generation and refinement, relative to the inference savings? Additionally, what explains the comparatively weaker performance on video retrieval and moment retrieval tasks in MMEB-v2?

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer xRZi (1/3)

Q1: Distinguishing latent reasoning from semantic representation learning.

We thank the reviewer for the discussion of latent computation and the criteria for interpreting it. ALICE aims to transfer useful, task-relevant information and structure from explicit CoT into intermediate representations for retrieval. Understanding the computational mechanisms underlying CoT-guided latent states remains an open research question [1, 2, 3]. We therefore approach this question empirically, combining representation analysis, controlled ablations, and interventions to examine what information these states encode and how they contribute to retrieval. Table 11 (originally Table 8) compares supervision using CoTs from other instances, global CoT embeddings, shuffled steps, and steps in their original order. These comparisons show that both the instance-specific content and the organization of supervision affect retrieval performance. The information analysis and interventions in Fig. 5 further show that the intermediate latents carry retrieval-relevant information and contribute to the final prediction.

We further evaluate ALICE on all 11 tasks of MRMR [4], a reasoning-intensive retrieval benchmark, to assess the practical benefits of CoT-guided latent computation.

MethodBackboneArtMedSciHumMathPhyEngBusNegDesignTrafficAVG(11)
OpenCLIPViT-G/14-1B56.017.933.222.05.75.07.09.713.08.112.417.3
PLUMEQwen2-VL-2B68.747.763.654.719.230.329.244.57.544.321.939.2
UME-R1Qwen2-VL-2B70.648.864.958.020.930.026.846.77.042.627.840.4
ALICEQwen2-VL-2B73.548.358.956.521.630.128.446.56.041.937.540.8
VLM2VecQwen2-VL-7B37.219.934.129.412.215.316.935.45.03.477.5319.7
UME-R1Qwen2-VL-7B77.855.772.964.127.239.232.247.87.561.941.748.0
ALICEQwen2-VL-7B75.754.368.360.131.143.134.053.09.570.040.349.0

With the same backbone, ALICE achieves average scores of 40.8 and 49.0 at the 2B and 7B scales, respectively, outperforming the corresponding methods with the same or comparable backbones in the table while retaining single-forward-pass inference without explicit CoT generation. These results further support ALICE's effectiveness in reasoning-intensive retrieval and its ability to generalize across benchmarks to some extent. These MRMR results have been added to Appendix B.3 / Table 10 of the revised manuscript.

[1] Are Latent Reasoning Models Easily Interpretable? COLM 2026.

[2] Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure. ICML 2026.

[3] Imagination Helps Visual Reasoning, But Not Yet in Latent Space. ICML 2026.

[4] MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval. ICLR 2026

AuthorsOpenReview ↗

Rebuttal to Reviewer xRZi (2/3)

Q2: The role of target-conditioned CoT supervision.

We use the ground-truth (GT) target during annotation to keep the generated explanations consistent with the query–target match, while requiring their content to be grounded in information available in the query (§3.2 and Appendix G). Using GT targets to guide CoT annotation is also adopted in related methods such as TTE, RGE, and DME.

To examine whether the benefit depends on target access during annotation, we regenerated the CoTs for the same 160K training subset without providing the target. We use Qwen2.5-VL-3B and evaluate after one epoch of S2, without subsequent S3 training.

Supervision settingMMEB Overall
No CoT alignment, with 15 latent tokens63.9
CoT generated without target access67.4
CoT generated with target access67.9

Without target access during annotation, CoT supervision still substantially improves over the unaligned baseline and achieves performance close to target-conditioned supervision. These results show that its benefits persist when the generator does not have access to the target.

The reference embedder uses target alignment to produce retrieval-relevant representations of the individual CoT steps. We examine whether these step-level anchors provide useful supervision beyond a shared target representation, and whether their organization matters. In Table 5, ALICE achieves 71.9, compared with 70.2 for direct target-embedding alignment. On the 160K training subset, Table 11 (originally Table 8) further compares direct target embeddings (66.1), global CoT embeddings (66.0), shuffled CoT steps (66.7), and steps in their original order (67.9). Together, these comparisons show that both step-level semantic content and its organization provide additional value for retrieval beyond direct target alignment.

Regarding individual latent steps, each latent group receives supervision from its corresponding CoT segment during S2. During S3, the alignment constraint is removed, allowing these intermediate representations to adapt to the retrieval objective (§3.5). We assess the retained correspondence by matching each latent group to the five CoT step anchors of the same instance using cosine similarity. On the 4,000 examples analyzed, matching accuracy is 59.92% after S2 and 54.60% after S3, indicating that step correspondence remains observable after free-form adaptation. We therefore interpret the final latent groups as CoT-guided, retrieval-adapted intermediate states. The interventions in Fig. 5 further support their contribution to the final retrieval decision.

AuthorsOpenReview ↗

Rebuttal to Reviewer xRZi (3/3)

Q3: Practical trade-offs and generalization.

Appendix D reports the model training costs, including the reference embedder. For Qwen2.5-VL-3B with K = 5, reference training, latent alignment, and free-form training together require 312 GPU-hours, compared with approximately 1,350 GPU-hours for our COCONUT-like implementation. Because CoT generation and refinement were performed through an internal API platform, their GPU-hour cost is difficult to quantify.

However, model training and CoT annotation are one-time offline investments for deployment. The step-level CoT annotations can also be reused across models and training runs, providing value beyond the present method. We plan to release these annotations together with the generation and refinement procedures to support further research on CoT-guided multimodal representations.

For video performance, our main hypothesis concerns the types of supervision provided by the training data. The video portion of the current MMEB-v2 training set mainly covers QA, caption retrieval, and text-to-video retrieval. Moment retrieval additionally requires precise localization of the temporal interval relevant to a query. The current training tasks provide limited direct supervision for this temporal localization requirement, making task coverage a plausible explanation for the observed differences in performance.

As reported in our response to Reviewer FSQ3, Q2, we retrained ALICE on UME-R1's released SFT dataset, using its query-side CoTs for supervision across reference training, S2, and S3. With Qwen2-VL-2B, the overall score remains 65.0: the image average changes from 71.1 to 70.5 and the VisDoc average from 72.6 to 71.8, while the video average improves by 2.0 points, from 42.7 to 44.7 (Appendix B.1 / Table 8). These results show that ALICE can effectively use CoT supervision beyond our own annotations. The improvement on video tasks also prompted us to examine differences in the supervision provided by the two annotation sources. We therefore compared annotation lengths on matched video examples and found that UME-R1's complete query-side annotations are on average 2.36× as long as ALICE's CoT annotations. These longer annotations may contain finer-grained video details, suggesting that annotation detail could be an additional factor affecting video performance.

Therefore, a promising future direction would be to incorporate explicit supervision for temporal localization and cross-frame event relations to help intermediate representations capture finer-grained temporal information.

Thank you for the detailed response and clarifications. The authors have addressed most of my concerns. I appreciate the additional explanations. I will raise my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Final Remarks

We thank the reviewers for their thoughtful feedback, constructive suggestions, and engagement with our responses. We also thank the Area Chair for their time and effort. We summarize the strengths highlighted by the reviewers, our responses to their main concerns, and the changes we made to the manuscript as follows.

Strengths Highlighted by Reviewers

  • Clear and practical motivation. While latent reasoning can reduce inference cost, next-token-trained latents tend to predict CoT step-initial tokens, motivating direct stepwise alignment in ALICE.
  • Sound methodological design. Step-level CoT embeddings provide direct semantic supervision for intermediate latent states, and the subsequent contrastive training stage further adapts those states to retrieval.
  • Retrieval quality and efficiency. ALICE encodes queries in a single forward pass, achieving substantially lower latency than RGE and UME-R1 while maintaining strong retrieval performance.
  • Broad evaluation and systematic ablations. MMEB experiments cover seven 2B–8B backbones across three Qwen-VL generations. Ablations examine the training stages, alignment loss, and alignment targets.

Main Concerns and Responses

  • Epoch-matched baselines. With two training epochs, the standard embedder and unaligned-latent baseline score 66.7 and 67.0, respectively, both still below ALICE’s 71.9. [FSQ3 Q1, nSah W1]
  • Direct CoT distillation into the final embedding. With the same data, teacher, and training budget, direct distillation scores 59.9 with cosine loss and 61.2 with InfoNCE, while stepwise alignment of intermediate latents scores 67.9. [nSah Q1]
  • CoT step correspondence after free-form training. On sampled training examples, step-matching accuracy is 59.92% after alignment and 54.60% after free-form training, indicating that step-level correspondence persists. [xRZi Q2, nSah Q3]
  • Target access during CoT annotation. On MMEB, query-only annotation scores 67.4 overall, close to 67.9 with target-assisted annotation and above 63.9 for the unaligned-latent baseline. [xRZi Q2, FSQ3 Q4]
  • Reasoning-intensive retrieval on MRMR. Across all 11 tasks, ALICE scores 40.8 / 49.0 at 2B / 7B, compared with 40.4 / 48.0 for UME-R1 on the same backbones. [xRZi Q1, nSah Q5]
  • Comparison using UME-R1’s SFT data. Retraining ALICE on UME-R1’s public SFT data maintains its 65.0 MMEB-v2 overall score and raises the video average from 42.7 to 44.7. [FSQ3 Q2, xRZi Q3]

Manuscript Revisions

  1. Replaced two Qwen2-VL rows in Table 1 with the corresponding image-only training results and expanded MMEB-v1/v2 comparisons with TTE, MMEmb-R1, LaME, and TTE-Flash.
  2. Added MRMR results, the UME-R1 data experiment, three-seed MMEB-v2 results, and the cosine distillation control to the appendix. Figure 4 now includes three-seed error bars for all four latent budgets.
  3. Refined the abstract, introduction, and conclusion to clarify the contribution of stepwise CoT supervision to retrieval representations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.