acceptodds
Under review as a conference paper at ICLR 2027

Reasoning Quality Matters: Combating Reasoning Collapse in LLM-based Embedding Learning

Abstract

Large Language Models (LLMs) have recently shown strong potential for producing context-rich text embeddings for retrieval. Most existing methods either treat embedding learning as passive feature extraction or exploit LLM reasoning through instruction following for better embedding optimization. However, specialization toward embedding objectives can suppress useful reasoning generation or produce retrieval-irrelevant text. We refer to these two forms of degradation as reasoning collapse. To address this issue, we propose CoFree (Collapse-Free Reasoning Embedding), a two-stage framework that progressively integrates LLM reasoning into query and document embedding optimization while preserving reasoning quality. At the first stage, CoFree applies reference-guided supervised fine-tuning to restore the reasoning ability and retain representational strength of the foundation embedding model. At the second stage, we introduce dual rewards, an embedding-oriented reward and a reasoning-oriented reward, to guarantee fine-grained reasoning of the relevance toward the embedding goal in reinforcement learning. This endpoint-coupled optimization transforms embedding learning from static alignment into a high-quality reasoning-guided search process for retrieval. Extensive experiments demonstrate the effectiveness of CoFree, with CoFree-4B achieving an average absolute improvement of 2.8 nDCG@10 points over Qwen3-Embedding-4B across 22 datasets from MTEB and BRIGHT. Online experiments in a real-world retrieval system further show consistent gains. Code, RTED, and model checkpoints will be made publicly available.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
text embeddings, dense retrieval, large language model reasoning, reinforcement learning, representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_xCfvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose CoFree, an LLM embedder that first generates a short "reasoning" text for a query or document and then embeds the input together with that text. The authors argue that such models suffer from "reasoning collapse". Embedding training either destroys the ability to generate text (generation collapse) or yields fluent text that does not help retrieval (semantic collapse).

CoFree starts from an existing embedding model (here, Qwen3-Embedding-4B or gte-Qwen2-1.5B-instruct) and trains it in two stages:

  1. SFT on reasoning texts that Qwen3-235B writes for queries and documents. The loss combines next-token prediction, InfoNCE, and an MSE term. The MSE term pulls the embedding of input plus reasoning toward the frozen backbone's embedding of the input excl. reasoning.
  2. GRPO with two rewards: the policy's own cosine margin between the positive and a hard negative, and the score margin of Qwen3-Reranker-4B on the reasoning-augmented pairs. A second MSE term anchors the query embeddings to the SFT model.

The authors benchmark on MTEB's retrieval tasks and BRIGHT. CoFree-4B reaches 40.8 vs. 38.0 for Qwen3-Embedding-4B. CoFree-1.5B reaches 34.6 vs. 26.9 for gte-Qwen2-1.5B-instruct.

I for now lean toward rejection. Reason-then-embed models are a crowded field (LREM, O1 Embedder, GRACE, UME-R1, Search-R3), and each component of CoFree has a close precedent. An additional contribution over those requires a clean experimental setup to show that its additions matter. This submission does not yet meet that bar. Notably, the backbones are evaluated well below their official scores, and the control that isolates the claimed mechanism is missing (cf. weaknesses). I invite the authors to rebut my review.

Critical strengths
  • Speed: CoT embedders are typically slow at inference, since they generate hundreds of tokens per query on average. In contrast to prior models, CoFree only needs 44 to 46.
  • Strong ablation: The authors input the the same generated text to six frozen encoders, with Echo and length-matched noise as controls. CoFree's text improves performance on every encoder, strongly supporting the notion that the generated text carries retrieval-relevant content.
Critical weaknesses

LLM backbones are listed below their official scores. On the same 10 retrieval tasks as Table 8, the official MTEB results of Qwen3-Embedding-4B average ~68.5, vs. 64.1 in Table 1. CoFree-4B (66.4) thus falls 2.1 points below its own backbone and wins only on sensitive ArguAna and HotpotQA. On the same tasks, gte-Qwen2-1.5B-instruct averages ~50.3 officially vs. 43.1 in Table 1.

This appears due to inconsistent handling of retrieval instructions (cf. ArguAna). The authors note that "Each model uses its own default retrieval instruction, fixed across all evaluation datasets without dataset-specific customization." This strongly suggests that CoFree's rewriting may recover the task information that a generic instruction removes.

The ablations do not isolate the claimed mechanism. The paper's own numbers point away from the reasoning and the shared generator-encoder:

  • The CoFree-1.5B encoder scores ~32.3 without reasoning and ~34.6 with it (cf. Table 14). Most of its almost 8-point gain over the backbone appears without reasoning
  • embeddinggemma-300m fed CoFree-1.5B's text reaches 36.20, above CoFree-1.5B itself
  • At 4B, contrastive and anchor training without reasoning scores 37.7, vs. 38.0 for the backbone. Contrastive-only FT drops to 32.6, so the "controlled comparisons" (L320) measure recovery from a damaging recipe
  • Removing the reasoning reward costs 0.5 points, comparable to the RL seed spread (40.7 to ~40.9)

Missing variants are: (i) the frozen backbones with official instructions plus teacher-, prompted-, or CoFree-generated text; (ii) the CoFree-4B encoder without and with teacher reasoning; (iii) RL with only KL and MSE, excl. both rewards.

Hence, soundness currently only scores as a 1.

Questions for authors
  1. Which instruction strings did each model, including CoFree, receive on each dataset? What do Qwen3-Embedding-4B, gte-Qwen2-1.5B-instruct and CoFree reach with the official task-specific instructions?
  2. What do the frozen LLMs reach when given CoFree-generated text on queries and documents? And what do they reach with text from the Qwen3-235B teacher (Prompts 1 and 2)? Table 14 runs this test for six other encoders, but not for the two backbones.
  3. What does the CoFree-4B encoder reach without reasoning at inference, and with teacher-written reasoning? This would separate the encoder training from the generated text, as Table 14 does for CoFree-1.5B.
12 comments
AuthorsOpenReview ↗

Response to Reviewer xCfv (1/4): Retrieval instructions

Thank you for recognizing CoFree's inference efficiency and the evidence that its generated text carries retrieval-relevant information across frozen encoders. We address the instruction comparison, the separate contributions of encoder training and inference reasoning, and the roles of the training objectives below.

Retrieval instructions and backbone scores (Weakness; Question 1): LLM backbones are listed below their official scores.

Which instruction strings did each model, including CoFree, receive on each dataset? What do Qwen3-Embedding-4B, gte-Qwen2-1.5B-instruct and CoFree reach with the official task-specific instructions?

Response: In Table 1, embedding models and CoFree use the respective fixed instructions listed below across all evaluation datasets. We use fixed instructions to support fair comparisons across datasets and reduce the influence of dataset-specific prompt choices on the reported gains. This also simplifies training on a mixture of retrieval datasets by allowing a common instruction format throughout training. In practical deployment, inputs may come from diverse or unfamiliar domains without a tailored task description, so we consider reliable retrieval with a general instruction an important aspect of robustness.

Fixed instructions used in Table 1:

ModelTable 1 instruction, fixed across datasets
Embedding-ModelRetrieval relevant passage for the given query.
CoFreeReason and Embed the input:

Results with dataset-specific instructions: Publicly reported scores and our re-evaluated scores may differ across evaluation setups. We compare the backbones and CoFree within the same MTEB evaluation framework, enabling the dataset-specific instructions provided by the repository for both. Scores are nDCG@10 (0–100).

ModelInstructionsMTEBBRIGHTOverall
Qwen3-Embedding-4BFixed (Table 1)64.116.338.0
Qwen3-Embedding-4BDataset-specific66.416.939.4
CoFree-4BFixed (Table 1)66.419.440.8
CoFree-4BDataset-specific67.021.041.9
gte-Qwen2-1.5B-instructFixed (Table 1)43.113.426.9
gte-Qwen2-1.5B-instructDataset-specific48.213.829.4
CoFree-1.5BFixed (Table 1)58.514.734.6
CoFree-1.5BDataset-specific59.616.536.1

Dataset-specific instructions improve all four models, while CoFree retains its advantage over both backbones. Under this matched setting, CoFree reaches 67.0 versus 66.4 on MTEB at 4B, and 59.6 versus 48.2 at 1.5B. It also improves BRIGHT and achieves Overall gains of 2.5 points at 4B (41.9 vs. 39.4) and 6.7 points at 1.5B (36.1 vs. 29.4). These results show that CoFree's gains persist when task information is explicitly supplied through dataset-specific instructions.

AuthorsOpenReview ↗

Response to Reviewer xCfv (2/4): Frozen-backbone reasoning

Frozen backbones with different reasoning sources (Question 2): What do the frozen LLMs reach when given CoFree-generated text on queries and documents? And what do they reach with text from the Qwen3-235B teacher (Prompts 1 and 2)? Table 14 runs this test for six other encoders, but not for the two backbones.

Response: We keep the two initialization backbones, Qwen3-Embedding-4B and gte-Qwen2-1.5B-instruct, fixed and compare retrieval performance using reasoning generated by CoFree-4B and the Qwen3-235B teacher. Teacher reasoning uses Prompts 1–2 in Appendix G.1 for query- and document-side generation, following the prompted-reasoning procedure in Appendix D.4.

Each sub-benchmark score is the mean nDCG@10 (0–100) across these two frozen encoders. The first two sub-benchmarks are from MTEB and the remaining three from BRIGHT. Mean averages the five sub-benchmark scores. All scores and gains are reported to one decimal place; means and gains are calculated before rounding.

Reasoning sourceCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theoremsMean
No reasoning48.271.715.116.915.333.4
Qwen3-235B47.165.122.414.818.633.6
CoFree-4B49.977.520.719.222.638.0

CoFree-4B reasoning improves the five-sub-benchmark mean by 4.6 points over no reasoning and 4.4 points over teacher reasoning. When averaged across the two backbones, it improves over no reasoning on all five sub-benchmarks and outperforms teacher reasoning on four of five. The teacher performs better on Biology.

We also compare Llama-3.2-3B-Instruct and CoFree-1.5B reasoning, reporting Overall nDCG@10 across all 22 datasets (Appendix D.4, Table 14).

Frozen encoderNo reasoningLlama reasoningCoFree reasoningGain over Llama
embeddinggemma-300m33.527.336.2+8.9
jina-embeddings-v330.725.832.9+7.0
Linq-Embed-Mistral31.826.534.8+8.3
Search-R3-Small26.023.627.8+4.2
CoFree-1.5B32.326.434.6+8.2
UME-R1-2B21.923.024.0+1.1

CoFree-1.5B reasoning outperforms Llama-generated reasoning on all six encoders, with gains of 1.1–8.9 Overall points. CoFree reasoning also improves over no reasoning on all six encoders.

AuthorsOpenReview ↗

Response to Reviewer xCfv (3/4): Encoder training and transfer

Encoder training versus inference reasoning (Question 3): What does the CoFree-4B encoder reach without reasoning at inference, and with teacher-written reasoning? This would separate the encoder training from the generated text, as Table 14 does for CoFree-1.5B.

The CoFree-1.5B encoder scores ~32.3 without reasoning and ~34.6 with it (cf. Table 14). Most of its almost 8-point gain over the backbone appears without reasoning.

Response: We hold the final CoFree-4B encoder fixed and compare original inputs, teacher reasoning, and CoFree reasoning on the same five selected sub-benchmarks. This separates the contribution of the generated input text from the performance already retained in the encoder's parameters. Scores are nDCG@10 (0–100), and Mean averages the five sub-benchmark scores.

Final encoderInference inputCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theoremsMean
CoFree-4BOriginal text56.287.420.217.627.741.8
CoFree-4BOriginal text + Qwen3-235B reasoning56.384.321.616.830.842.0
CoFree-4BOriginal text + CoFree reasoning57.888.722.418.532.644.0

With the final encoder held fixed, CoFree reasoning improves the five-sub-benchmark mean by 2.2 points over original inputs and 2.0 points over teacher reasoning. It outperforms both alternatives on all five sub-benchmarks, showing the additional retrieval benefit of CoFree's generated text.

Across all 22 datasets, the following Overall nDCG@10 results separate the gains from encoder training and inference reasoning.

ModelInference inputOverall
Qwen3-Embedding-4BOriginal text38.0
CoFree-4BOriginal text38.7
CoFree-4BOriginal text + CoFree reasoning40.8

Encoder training improves Overall by 0.7 points. Holding the final CoFree-4B encoder fixed, adding its reasoning further improves Overall by 2.1 points, with gains of 2.5 points on MTEB and 1.7 points on BRIGHT.

At 1.5B, the corresponding progression is 26.9 for the original backbone, 32.3 for the final CoFree encoder on original inputs, and 34.6 with CoFree reasoning. Thus, encoder training accounts for the larger part of the 1.5B improvement, while inference reasoning provides a further 2.3 points. The two scales demonstrate both improved encoder representations and an additional contribution from generated reasoning.

Transfer versus a shared generator–encoder (Weakness): embeddinggemma-300m fed CoFree-1.5B's text reaches 36.20, above CoFree-1.5B itself.

Response: EmbeddingGemma-300M already scores higher without reasoning: 33.50 versus 32.33 for the final CoFree-1.5B encoder. Adding the same CoFree-1.5B reasoning improves Gemma to 36.20 and CoFree to 34.61 (Appendix D.4, Table 14). Gemma's higher final score therefore builds on its stronger original-input performance, with CoFree reasoning improving both encoders.

CoFree's joint training enables a single model to generate retrieval-useful reasoning and produce embeddings. The gain on the stronger Gemma encoder further supports the quality of CoFree's reasoning text and its usefulness beyond CoFree's own encoder.

Recovery from contrastive fine-tuning (Weakness): At 4B, contrastive and anchor training without reasoning scores 37.7, vs. 38.0 for the backbone. Contrastive-only FT drops to 32.6, so the "controlled comparisons" (L320) measure recovery from a damaging recipe.

Response: Our SFT stage aims to restore reasoning generation while preserving the pretrained embedding model's representational ability (Section 3.1). Language-modeling supervision restores generation, while contrastive learning and reference guidance preserve embedding quality during this process.

We include Contrastive FT to rule out the possibility that the gains come solely from the training data. The 4B results are summarized below (Overall nDCG@10; Tables 1 and 4).

Model / training variantOverall
Original backbone (Qwen3-Embedding-4B)38.0
Contrastive FT32.6
Contrastive FT + reference guidance37.7
Full SFT39.9

Reference guidance retains retrieval performance close to the original backbone. Full SFT additionally learns reasoning generation and improves Overall by 2.2 points over the reference-guided control and 1.9 points over the original backbone.

Reviewer_XP3xOpenReview ↗
Rating
2
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper argues that LLM-based embedding models that generate “reasoning” text before embedding suffer from two failure modes, which it calls reasoning collapse: Generation collapse: embedding specialization destroys the model’s ability to generate fluent text. Semantic collapse: fluent reasoning that is generic or irrelevant to retrieval. It proposes CoFree, a two-stage method: Reasoning Restoration (SFT). Joint training with a language-modeling loss on teacher-generated reasoning, an InfoNCE loss, and a reference-guided MSE loss that anchors Emb (x⊕r) to the frozen backbone’s Emb(x). Reasoning Augmentation (RL). GRPO with two rewards: an embedding-margin reward (cosine similarity gap between positive and negative); a reranker-margin reward (Qwen3-Reranker-4B), plus an MSE regularizer toward the SFT model. The authors also construct RTED (3.6M query–document tuples with teacher reasoning from Qwen3-235B, filtered by rerankers and an embedding-similarity rule).

Critical strengths
  1. Relevant, well-motivated problem. Naively appending generated text to embedding inputs can hurt. The paper shows this with UME-R1 and prompted Llama reasoning in Table 14, and with CL-only training on reasoning-augmented inputs (28.5 overall, Table 17).

  2. Controls on the reasoning’s usefulness. The Echo and length-matched Noise controls, six frozen encoders, and a HyDE comparison (Table 14) are good designs for separating “extra tokens” from “useful content”.

  3. Honest reporting of weaknesses. Appendix C.2 openly discusses regressions on LeetCode and Pony. Section 4.4 reports three RL seeds. Appendix E.1 tests whether reasoning-augmented training without the LM loss recovers the gain.

Critical weaknesses
  1. The headline gain is not attributed to “reasoning quality”; key confounds are uncontrolled. The reasoning supervision comes from Qwen3-235B-A22B. No baseline has access to this knowledge. The largest BRIGHT gains are on the theorem subsets (TheoremQA-theorems 22.5 → 32.6), where teacher knowledge plausibly matters most.

  2. Data refinement, negative mining and the RL reward all rely on Qwen3-Reranker/Embedding models. The RL stage amounts to distilling a cross-encoder’s margin into the policy. There is no RL (or fine-tuning) control with the same reranker signal but without generated reasoning.

  3. The RL stage’s contribution is small and its mechanism is unclear.

Questions for authors
  1. What are the results for Qwen3-Embedding-4B and the other frozen encoders using reasoning generated by the teacher (Qwen3-235B) with Prompts 1–2, and with Qwen3-4B prompted similarly?

  2. Can you provide CoFree-4B without inference reasoning, and a control trained with the same reranker signal and data but no generated reasoning?

6 comments
AuthorsOpenReview ↗

Response to Reviewer XP3x (1/3): Teacher and reasoning sources

Thank you for recognizing the motivation of CoFree and the controls on the usefulness of generated reasoning. We address your attribution questions with direct comparisons against teacher-generated and Qwen3-4B reasoning, a control that distills the same reranker without reasoning, and the stage-wise retrieval and reasoning diagnostics.

Teacher knowledge and reasoning quality (Weakness 1): The headline gain is not attributed to “reasoning quality”; key confounds are uncontrolled. The reasoning supervision comes from Qwen3-235B-A22B. No baseline has access to this knowledge. The largest BRIGHT gains are on the theorem subsets, where teacher knowledge plausibly matters most.

Question 1: What are the results for Qwen3-Embedding-4B and the other frozen encoders using reasoning generated by the teacher (Qwen3-235B) with Prompts 1–2, and with Qwen3-4B prompted similarly?

Response: We keep the encoders fixed and compare retrieval performance using reasoning generated by CoFree-4B, its teacher Qwen3-235B, and Qwen3-4B.

We follow the prompted-reasoning protocol in Appendix D.4, using Prompts 1–2 in Appendix G.1 for query- and document-side generation. The two frozen encoders are Qwen3-Embedding-4B and EmbeddingGemma-300M. Each sub-benchmark score is the mean nDCG@10 (0–100) across these two encoders. The first two sub-benchmarks are from MTEB and the remaining three from BRIGHT.

Reasoning sourceCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theoremsMean
No reasoning47.682.421.216.615.336.6
Qwen3-4B46.067.522.916.524.835.5
Qwen3-235B48.169.429.117.221.437.1
CoFree-4B49.682.827.020.425.841.1

CoFree reasoning exceeds teacher reasoning by 4.0 points on the mean, with higher two-encoder average scores on four of five sub-benchmarks. Qwen3-235B performs better on Biology. On TheoremQA-theorems, which you specifically highlighted, teacher reasoning improves the score from 15.3 to 21.4, while CoFree reasoning reaches 25.8. Thus, the teacher's text helps on this task, and CoFree's learned reasoning provides a further retrieval benefit.

CoFree reasoning exceeds Qwen3-4B reasoning by 5.6 points on the mean and outperforms both Qwen3-4B reasoning and no reasoning on all five sub-benchmarks when averaged across the two encoders. Qwen3-4B reasoning improves over no reasoning on Biology and TheoremQA-theorems.

We also compare Llama-3.2-3B-Instruct and CoFree-1.5B reasoning, reporting Overall nDCG@10 across all 22 datasets (Appendix D.4, Table 14). Llama reasoning is generated with the same query- and document-side RTED prompts (Prompts 1–2).

Frozen encoderNo reasoningLlama reasoningCoFree reasoningGain over Llama
embeddinggemma-300m33.527.336.2+8.9
jina-embeddings-v330.725.832.9+7.0
Linq-Embed-Mistral31.826.534.8+8.3
Search-R3-Small26.023.627.8+4.2
CoFree-1.5B32.326.434.6+8.2
UME-R1-2B21.923.024.0+1.1

CoFree-1.5B reasoning outperforms Llama-generated reasoning on all six encoders, with gains of 1.1–8.9 Overall points. CoFree reasoning also improves over no reasoning on all six encoders.

AuthorsOpenReview ↗

Response to Reviewer XP3x (2/3): Distillation and inference ablation

Same reranker supervision without reasoning (Weakness 2): Data refinement, negative mining and the RL reward all rely on Qwen3-Reranker/Embedding models. The RL stage amounts to distilling a cross-encoder’s margin into the policy. There is no RL (or fine-tuning) control with the same reranker signal but without generated reasoning.

Question 2, training control: Can you provide a control trained with the same reranker signal and data but no generated reasoning?

Response: We include direct reranker distillation into embeddings, with no reasoning during training or inference. To control for data filtering and hard-negative selection, Contrastive FT uses the same retained training tuples, selected negatives, and backbone initialization as CoFree. We additionally combine contrastive learning with KL distillation using scores from Qwen3-Reranker-4B, the same model used for our RL reward.

For each query, the embedding model scores candidate documents by cos⁡(Emb(q),Emb(d))\cos(\mathrm{Emb}(q),\mathrm{Emb}(d)), while the frozen reranker scores each original pair (q,d)(q,d) directly. We apply softmax to both score vectors and minimize DKL(preranker∣pembedding)D_{\mathrm{KL}}(p_{\mathrm{reranker}}\\|p_{\mathrm{embedding}}), following the KL distillation objective for dense retrieval in [1]. Both distillation variants use the same training examples and Qwen3-Embedding-4B initialization. The second also retains the reference-guided loss LRGL_{\mathrm{RG}} (Eq. 4), with γ=40\gamma=40.

Scores are nDCG@10 (0–100). Overall averages all 22 datasets: 10 from MTEB and 12 from BRIGHT.

MethodMTEBBRIGHTOverall
Contrastive FT on the same data50.917.332.6
Contrastive FT + reranker KL distillation52.416.933.0
Contrastive FT + reranker KL distillation + LRGL_{\mathrm{RG}}59.118.336.8
CoFree-4B66.419.440.8

CoFree exceeds the regularized distillation baseline by 7.3 points on MTEB, 1.1 points on BRIGHT, and approximately 4.0 points Overall. This comparison supports the benefit of CoFree over the tested alternatives that directly transfer the same reranker's supervision into embeddings. In CoFree, that supervision rewards generated text according to its ability to separate relevant from irrelevant documents; the embedding reward additionally optimizes discrimination in the representation space.

Question 2, inference ablation: Can you provide CoFree-4B without inference reasoning?

Response: We hold the final CoFree-4B encoder fixed and compare original inputs with reasoning-augmented inputs. Scores are nDCG@10.

Final encoderInference inputMTEBBRIGHTOverall
CoFree-4BOriginal text63.917.738.7
CoFree-4BOriginal text + CoFree reasoning66.419.440.8

Adding CoFree reasoning improves MTEB by 2.5 points, BRIGHT by 1.7 points, and Overall by 2.1 points, showing its additional retrieval benefit at inference.

AuthorsOpenReview ↗

Response to Reviewer XP3x (3/3): RL contribution and mechanism

RL contribution and mechanism (Weakness 3): The RL stage’s contribution is small and its mechanism is unclear.

Response: RL's contribution. At 4B, RL adds 0.9 Overall nDCG@10 points beyond SFT, averaged over three RL runs (Table 4). This gain is measured across 22 datasets: 10 MTEB datasets for general retrieval and 12 BRIGHT datasets for reasoning-intensive retrieval.

RL also improves the retrieval relevance of generated reasoning. For CoFree-1.5B, mean positive–negative discrimination accuracy rises from 65.6% after SFT to 73.0% after RL, a gain of 7.4 percentage points. These scores are averaged over BGE, Mixedbread, and Jina rerankers, all distinct from the training reward model (Section 4.3; Appendix D.5, Table 16).

Mechanism. RL rewards reasoning that increases the positive–negative relevance margin, using both reranker scores and embedding similarities.

The LLM Judge scores for CoFree-1.5B are shown below (Appendix D.6, Table 17; scale 1–5). The judges assess intent preservation, reasonable and accurate expansion, and specific information contribution.

CoFree-1.5B configurationQwen3.5-27BGLM-4-32BHunyuan-A13B
Without reasoning reward3.13.93.9
Full RL3.34.14.1

Full RL receives higher reasoning-content scores from all three judges, supporting the contribution of the reasoning reward.

The fixed-encoder comparisons above show that CoFree reasoning produces higher mean retrieval scores than teacher or Qwen3-4B reasoning when supplied to the same encoders. The final CoFree-4B inference ablation further shows a 2.1-point Overall gain from adding its reasoning. The SFT-to-RL diagnostic shows the improvement in reasoning relevance during RL, while the fixed-encoder experiments establish the contribution of the generated text to retrieval performance.

References

[1] Lin, Sheng-Chieh, Jheng-Hong Yang, and Jimmy Lin. "In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval." Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021). 2021.

Reviewer_Knp4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies reasoning collapse in embedding models for reasoning-intensive retrieval, encompassing degraded generation capabilities and insufficiently discriminative reasoning content. To address this issue, it proposes a two-stage framework that preserves reasoning capabilities during embedding specialization. The first stage uses multi-objective SFT to restore reasoning and generation abilities through teacher-generated supervision, while maintaining representation quality through contrastive learning and reference-guided alignment with the backbone’s embeddings. The second stage employs a GRPO variant with dual rewards to improve the retrieval utility of generated reasoning, alongside embedding regularization to preserve representation quality. Experiments on public benchmarks and in a real-world retrieval system demonstrate the effectiveness of the approach.

Critical strengths
  1. The paper is well motivated, identifying reasoning collapse in reasoning-intensive retrieval methods. The proposed approach is intuitively aligned with the identified problems.
  2. The experimental evaluation is extensive, including deployment in a real-world retrieval system and further analysis of the quality of CoFree’s reasoning outputs.
Critical weaknesses
  1. Most of the reasoning-based embedding baselines are designed for multimodal retrieval, including UME-R1-2B and TWN-4B, and may not be specifically optimized for dense text retrieval. I believe important methods such as ReasonEmbed[1], ReasonIR[2], and RaDeR[3] should be considered as baselines to better establish the advantages of the proposed approach. Moreover, several important empirical findings remain insufficiently explained, such as why reasoning-based baselines underperform some non-reasoning baselines overall and why Contrastive FT-4B performs worse than its initialization backbone, Qwen3-Embedding-4B.
  2. The necessity of reinforcement learning in Stage 2 remains insufficiently justified, both conceptually and empirically, relative to simpler alternatives. Stage 2 aims to improve the quality of reasoning generated by the embedding model. However, using Qwen3-4B without embedding specialization to generate reasoning text under a matched computational budget could offer a simpler and potentially effective alternative.

[1] Chen, Jianlyu, et al. "Reasonembed: Enhanced text embeddings for reasoning-intensive document retrieval." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

[2] Shao, Rulin, et al. "Reasonir: Training retrievers for reasoning tasks." arXiv preprint arXiv:2504.20595 (2025).

[3] Das, Debrup, Sam O’Nuallain, and Razieh Rahimi. "Rader: Reasoning-aware dense retrieval models." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Questions for authors
  1. Could the authors explain why the reasoning-based embedding baselines underperform the non-reasoning baselines overall?
  2. Could the authors explain why Contrastive FT-4B performs worse than Qwen3-Embedding-4B after additional fine-tuning?
  3. Could the authors provide a controlled comparison demonstrating the advantage of Stage 2 over directly using reasoning text generated by Qwen3-4B, together with a conceptual explanation of the observed differences?
7 comments
AuthorsOpenReview ↗

Response to Reviewer Knp4 (1/2): Baselines and contrastive fine-tuning

Thank you for recognizing the motivation of CoFree and our analyses of reasoning quality. We have added the requested text-retrieval baselines and a direct comparison with Qwen3-4B reasoning, and provide further evidence on the roles of continued fine-tuning and Stage 2.

Text-retrieval baselines (Weakness 1): Most of the reasoning-based embedding baselines are designed for multimodal retrieval, including UME-R1-2B and TWN-4B, and may not be specifically optimized for dense text retrieval.

I believe important methods such as ReasonEmbed[1], ReasonIR[2], and RaDeR[3] should be considered as baselines to better establish the advantages of the proposed approach.

Response: Our evaluation already includes Search-R3 [4], a text-retrieval method. Like CoFree, it first generates reasoning and then extracts embeddings using the same model.

We have evaluated all three requested baselines using the MTEB framework. These methods focus on constructing in-domain training data that favors performance on BRIGHT [1–3]. In particular, ReasonEmbed directly uses BRIGHT's document corpora to construct its training data [1].

CoFree focuses on preserving reasoning and embedding quality during training, using the diverse retrieval sources listed in Appendix A.1. We additionally continue training the final CoFree-4B model on one quarter of ReasonEmbed's data and evaluate the resulting model on both BRIGHT and MTEB.

Scores are nDCG@10 (0-100). Overall averages all 22 datasets: 10 from MTEB and 12 from BRIGHT.

ModelMTEBBRIGHTOverall
ReasonIR-8B26.920.523.4
RaDeR-3B41.218.328.7
ReasonEmbed-4B23.232.528.3
CoFree-4B66.419.440.8
CoFree-4B + ReasonEmbed data47.227.236.3

CoFree's main-paper results have the highest MTEB and Overall scores in this comparison, while ReasonEmbed leads on BRIGHT.

The continued-training experiment shows that a BRIGHT gain can come at a substantial cost to general retrieval. Starting from the final CoFree-4B model, training on ReasonEmbed's in-domain data improves BRIGHT performance, reaching 27.2, but reduces MTEB from 66.4 to 47.2, a 19.2-point drop. This supports training-data specialization as one reason why stronger reasoning-intensive retrieval does not necessarily translate into stronger overall retrieval.

Performance of reasoning-based models (Question 1): Could the authors explain why the reasoning-based embedding baselines underperform the non-reasoning baselines overall?

Response: The performance gap is already present without reasoning. The fixed-encoder controls in Appendix D.4 report the following Overall nDCG@10 scores.

EncoderNo reasoningOwn reasoning
EmbeddingGemma-300M33.5—
Search-R3-Small26.025.8
UME-R1-2B21.922.1

Their own reasoning does not close this gap. Search-R3-Small and UME-R1-2B already underperform EmbeddingGemma-300M without reasoning. Adding their own reasoning slightly lowers Search-R3's score and only marginally improves UME-R1's, leaving both below EmbeddingGemma-300M.

CoFree reasoning improves Overall by 1.8-2.9 points across all six fixed encoders, showing that improving reasoning quality can increase retrieval performance with the same encoder.

Contrastive fine-tuning (Question 2): Could the authors explain why Contrastive FT-4B performs worse than Qwen3-Embedding-4B after additional fine-tuning?

Response: We attribute the drop to limitations in our continued fine-tuning data and training setup, together with the lack of an explicit constraint to preserve the backbone's representations. Qwen3-Embedding uses approximately 150 million multilingual, multi-task synthetic pairs followed by high-quality supervised fine-tuning [5], whereas our continued fine-tuning uses 3.6 million retrieval examples. Qwen3-Embedding's full training data and original training configurations are not publicly available, so we use our own data and training setup.

Training-pair quality, batch size, and negative sampling affect the strength of contrastive supervision. Lower-quality pairs or suboptimal settings can weaken this supervision and degrade the backbone's existing representations during further fine-tuning. Our Contrastive FT control uses the same initialization, query-document pairs, and training settings as CoFree-SFT, with only the contrastive objective (Appendix B). It improves BRIGHT from 16.3 to 17.3 but reduces MTEB from 64.1 to 50.9 and Overall from 38.0 to 32.6 (Table 1).

Under the same training conditions, reference guidance raises Overall from 32.6 to 37.7, recovering most of the gap to the backbone's 38.0 (Table 4). This directly supports the role of preserving pretrained embedding quality during continued fine-tuning.

AuthorsOpenReview ↗

Response to Reviewer Knp4 (2/2): Stage 2 and reasoning quality

Stage 2 and a general-purpose reasoner (Weakness 2): The necessity of reinforcement learning in Stage 2 remains insufficiently justified, both conceptually and empirically, relative to simpler alternatives. Stage 2 aims to improve the quality of reasoning generated by the embedding model. However, using Qwen3-4B without embedding specialization to generate reasoning text under a matched computational budget could offer a simpler and potentially effective alternative.

Question 3: Could the authors provide a controlled comparison demonstrating the advantage of Stage 2 over directly using reasoning text generated by Qwen3-4B, together with a conceptual explanation of the observed differences?

Response: General-purpose generation ability does not by itself ensure that the generated reasoning improves retrieval. Stage 1 restores reasoning generation while preserving embedding quality. Stage 2 aims to further improve final retrieval performance by enhancing the quality and retrieval utility of generated reasoning through reinforcement learning.

We directly compare CoFree-4B reasoning with Qwen3-4B reasoning using the prompted-reasoning protocol in Appendix D.4 and the generation prompts in Appendix G.1. We keep Qwen3-Embedding-4B and EmbeddingGemma-300M fixed and vary the reasoning source. Each sub-benchmark score is the mean nDCG@10 (0–100) across these two encoders. The first two sub-benchmarks are from MTEB and the remaining three from BRIGHT. Mean averages the five sub-benchmark scores. All scores and gains are reported to one decimal place; means and gains are calculated before rounding.

Reasoning sourceCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theoremsMean
No reasoning47.682.421.216.615.336.6
Qwen3-4B46.067.522.916.524.835.5
CoFree-4B49.682.827.020.425.841.1

CoFree reasoning improves the mean score from 35.5 to 41.1 compared with Qwen3-4B reasoning, a gain of 5.6 points. It outperforms both Qwen3-4B reasoning and no reasoning on all five sub-benchmarks when averaged across the two encoders, whereas Qwen3-4B reasoning improves over no reasoning on Biology and TheoremQA-theorems.

Qwen3.5-27B also evaluates the reasoning quality of these two models on the same five sub-benchmarks, following Appendix D.6 (Prompt 7).

Reasoning sourceCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theorems
Qwen3-4B3.03.13.13.13.5
CoFree-4B3.64.04.04.03.6

CoFree-4B receives higher reasoning-quality scores on all five sub-benchmarks. Together with the retrieval results, this supports the advantage of CoFree reasoning over directly using Qwen3-4B as the reasoner. The additional benefit of Stage 2 is shown in Table 4: Overall increases from 39.9 after Full SFT to 40.8 after Full RL (averaged over three RL runs).

We also compare Llama-3.2-3B-Instruct and CoFree-1.5B reasoning, reporting Overall nDCG@10 across all 22 datasets (Appendix D.4, Table 14).

Frozen encoderNo reasoningLlama reasoningCoFree reasoningGain over Llama
embeddinggemma-300m33.527.336.2+8.9
jina-embeddings-v330.725.832.9+7.0
Linq-Embed-Mistral31.826.534.8+8.3
Search-R3-Small26.023.627.8+4.2
CoFree-1.5B32.326.434.6+8.2
UME-R1-2B21.923.024.0+1.1

CoFree-1.5B reasoning outperforms Llama-generated reasoning on all six encoders, with gains of 1.1–8.9 Overall points. CoFree reasoning also improves over no reasoning on all six encoders.

References

[1] Chen, Jianlyu, et al. "Reasonembed: Enhanced text embeddings for reasoning-intensive document retrieval." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

[2] Shao, Rulin, et al. "Reasonir: Training retrievers for reasoning tasks." arXiv preprint arXiv:2504.20595. 2025.

[3] Das, Debrup, Sam O'Nuallain, and Razieh Rahimi. "Rader: Reasoning-aware dense retrieval models." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

[4] Gui, Yuntao, and James Cheng. "Search-R3: Unifying Reasoning and Embedding Generation in Large Language Models." arXiv preprint arXiv:2510.07048. 2025.

[5] Zhang, Yanzhao, et al. "Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models." arXiv preprint arXiv:2506.05176. 2025.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer Knp4,

I hope this message finds you well. With less than three days remaining in the discussion period, we would like to follow up on our response, particularly the requested retrieval baselines, the comparison with Qwen3-4B reasoning, and the explanations of continued fine-tuning and Stage 2.

We would be grateful to hear whether these additional results and clarifications address your concerns. If any questions remain, please let us know; we would be happy to discuss them further.

Thank you for your time and effort in reviewing our paper.

Best regards,
The authors

Reviewer_JjBDOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper identifies reasoning collapse as a key challenge when integrating reasoning text generation into LLM-based embedding models. The authors distinguish between two types: generation collapse, where embedding training degrades the model’s generation capabilities, and semantic collapse, where the model generates fluent but irrelevant or unhelpful content for retrieval.

To address these issues, the authors propose a two-stage framework called CoFree.

  • Stage 1 (SFT): The model is trained on reasoning text generated by Qwen3-235B using a combination of language modeling loss, contrastive loss, and a reference-guided loss that encourages embeddings augmented with reasoning to remain aligned with the original backbone embeddings, thereby restoring generation capabilities.
  • Stage 2 (RL): GRPO is used to jointly optimize rewards based on embedding similarity margins and relevance margins scored by Qwen3-Reranker-4B. An additional MSE regularization term is applied to preserve the embedding space. The authors construct RTED, a training dataset containing 3.6 million examples. Across 22 datasets, including 10 English retrieval tasks from MTEB English v2 and 12 tasks from BRIGHT, CoFree-4B reportedly improves average nDCG@10 by 2.8 points over its Qwen3-Embedding-4B backbone, increasing the score from 38.0 to 40.8. The paper also presents results from an online A/B test conducted in a real-world e-commerce search system.
Critical strengths
  1. Comprehensive validation of reasoning text quality. In transfer experiments using six fixed encoders and varying only the reasoning text (Table 14), the authors include both an Echo baseline that repeats the input and a length-matched Noise baseline. CoFree reasoning consistently improves performance by 1.75–2.94 points over the original inputs across all encoders. The authors also evaluate discrimination accuracy using three rerankers distinct from the reward model (65.60% for SFT vs. 72.97% for RL, dropping to 64.47% when the reasoning reward is removed). Furthermore, they directly track the collapse of generation capabilities during contrastive training using Gemma-2-2B, with GSM8K accuracy falling from 41.93% to 0%. These experiments provide concrete evidence supporting the paper's claims.

  2. Practicality and reproducibility. CoFree generates relatively short reasoning traces (approximately 44 tokens on average), achieving 3.6–14× higher throughput than existing reasoning-based embedding methods. By preserving the original backbone architecture, it also remains compatible with inference engines such as vLLM. The RL stage is evaluated across three random seeds, and the authors commit to releasing the code, data, and checkpoints.

Critical weaknesses

The source of performance gains remains unclear: methodological improvements cannot be disentangled from distillation of a strong teacher model and potential data overlap.

  • Knowledge from strong teacher models is introduced through multiple pathways. Reasoning traces are generated by Qwen3-235B, hard-negative selection and positive-example filtering are handled by Qwen3-Reranker-8B and Qwen3-Reranker-0.6B, and RL rewards are provided by Qwen3-Reranker-4B. In particular, the reasoning reward effectively distills reranker scores. However, the paper lacks a baseline that directly distills the same reranker scores into embeddings without reasoning. Consequently, it remains unclear whether the observed gains stem from the proposed reasoning mechanism or simply from distilling the reranker's knowledge.

I find the core idea very interesting: the paper identifies reasoning collapse as a concrete failure mode in reasoning-augmented embedding models and directly measures it, rather than assuming that generated reasoning is always helpful. The experimental work is also thorough and practically grounded. The stage-wise analyses from SFT through RL clearly show how each component affects both retrieval and reasoning quality, and the online A/B test in a production e-commerce search system provides valuable evidence of real-world effectiveness, which is rarely seen in this line of work.

Questions for authors

Please refer to weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer JjBD (1/1): Teacher knowledge and direct distillation

Thank you for your thoughtful review and for recognizing our reasoning-quality analyses and practical evaluation. We appreciate your suggestion to clarify the source of the gains through direct reranker distillation.

Weakness — Source of the gains: The source of performance gains remains unclear: methodological improvements cannot be disentangled from distillation of a strong teacher model and potential data overlap.

Knowledge from strong teacher models is introduced through multiple pathways. Reasoning traces are generated by Qwen3-235B, hard-negative selection and positive-example filtering are handled by Qwen3-Reranker-8B and Qwen3-Reranker-0.6B, and RL rewards are provided by Qwen3-Reranker-4B.

Response: To control for data filtering and hard-negative selection, Contrastive FT uses the same retained training tuples, selected negatives, and backbone initialization as CoFree. At 4B, it achieves 32.6 Overall versus 40.8 for CoFree (Table 1), showing the benefit of our training framework over contrastive fine-tuning on the same data.

For train–test overlap, Appendix A.1 specifies that we remove training instances whose lowercased query or positive document exactly matches the corresponding evaluation text.

To compare CoFree reasoning with Qwen3-235B teacher reasoning, we follow the prompted-reasoning protocol in Appendix D.4, using the generation prompts in Appendix G.1. We keep Qwen3-Embedding-4B and EmbeddingGemma-300M fixed and vary the reasoning source. Each sub-benchmark score is the mean nDCG@10 (0–100) across these two encoders. The first two sub-benchmarks are from MTEB and the remaining three from BRIGHT. Mean averages the five sub-benchmark scores. All scores and gains are reported to one decimal place.

Reasoning sourceCQADupstackUnixTRECCOVIDBiologySustainable LivingTheoremQA-theoremsMean
No reasoning47.682.421.216.615.336.6
Qwen3-235B48.169.429.117.221.437.1
CoFree-4B49.682.827.020.425.841.1

CoFree reasoning exceeds teacher reasoning by 4.0 points on the five-sub-benchmark mean, with higher two-encoder average scores on four of five sub-benchmarks; Qwen3-235B performs better on Biology. This supports the retrieval utility of CoFree reasoning relative to directly using teacher-generated text on these tasks.

Weakness — Direct reranker distillation: In particular, the reasoning reward effectively distills reranker scores. However, the paper lacks a baseline that directly distills the same reranker scores into embeddings without reasoning. Consequently, it remains unclear whether the observed gains stem from the proposed reasoning mechanism or simply from distilling the reranker's knowledge.

Response: We combine contrastive learning with KL distillation using scores from Qwen3-Reranker-4B, the same model used for our RL reward. For each query, the embedding model scores candidate documents by cos⁡(Emb(q),Emb(d))\cos(\mathrm{Emb}(q),\mathrm{Emb}(d)), while the frozen reranker scores each original pair (q,d)(q,d) directly. We apply softmax to both score vectors and minimize DKL(preranker∣pembedding)D_{\mathrm{KL}}(p_{\mathrm{reranker}}\\|p_{\mathrm{embedding}}), following the KL distillation objective for dense retrieval in [1].

Both variants use the same training examples and Qwen3-Embedding-4B initialization, without reasoning during training or inference. The second variant also retains the reference-guided loss LRGL_{\mathrm{RG}} (Eq. 4), with γ=40\gamma=40.

MethodMTEBBRIGHTOverall
Contrastive FT on the same data50.917.332.6
Contrastive FT + reranker KL distillation52.416.933.0
Contrastive FT + reranker KL distillation + LRGL_{\mathrm{RG}}59.118.336.8
CoFree-4B66.419.440.8

CoFree exceeds the regularized distillation baseline by 7.3 points on MTEB, 1.1 points on BRIGHT, and approximately 4.0 points Overall. This supports CoFree's benefit over the tested direct score-distillation alternatives.

[1] Lin, Sheng-Chieh, Jheng-Hong Yang, and Jimmy Lin. "In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval." Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021). 2021.

AuthorsOpenReview ↗

Thank you and follow-up on our response

Dear Reviewer JjBD,

Thank you again for your thoughtful review and constructive suggestions. Your suggestion to include direct reranker distillation helped us clarify the source of CoFree's gains. We have included this comparison in our response, alongside the teacher-reasoning results and the clarification of train–test overlap filtering.

As the discussion period draws to a close, we would welcome any further feedback you may have on these additions. We would be happy to clarify any remaining points.

Thank you again for your time and careful consideration.

Best regards,
The authors

Thank you for the detailed response and additional experiments.

I particularly appreciate the inclusion of the direct distillation baseline using the same Qwen3-Reranker-4B. CoFree outperforms this baseline in Overall, providing stronger evidence for the effectiveness of the proposed reasoning-based training framework.

The comparison with Qwen3-235B teacher reasoning and the Contrastive FT baseline using the same training data also help address my initial concerns.

Overall, the additional experiments substantially address my concerns, and I find the authors' responses convincing. I appreciate their thorough efforts and maintain my positive assessment of the paper.

AuthorsOpenReview ↗

Thank you for the constructive discussion

Dear Reviewer JjBD,

Thank you for your thoughtful follow-up and continued positive assessment. We are glad that the additional comparisons helped address your concerns.

Your suggestion to include direct distillation from the same reranker helped us clarify the source of CoFree's gains and strengthen the experimental evidence. We appreciate your careful review and constructive feedback throughout the discussion.

Best regards,
The authors

Reviewer_yBBgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the degradation of generated reasoning in LLM-based embedding models and its implications for retrieval. The authors propose CoFree, a two-stage framework that uses SFT to restore reasoning generation while preserving embedding quality, followed by RL with rewards aimed to improve both embedding and reasoning quality. The authors report improvements in average retrieval performance on MTEB and BRIGHT, supported by ablations and additional analyses of reasoning quality.

Critical strengths
  • I find the problem well motivated and relevant, particularly the challenge of maintaining useful reasoning while optimizing embeddings for retrieval.
  • I find the experimental evaluation relatively broad, which covers multiple benchmarks and model sizes and includes a range of ablations and additional analyses.
  • I also find the paper well written and easy to follow, with a clear progression from the motivation to the proposed method and experimental evaluation.
Critical weaknesses
  • The additional gain from RL at 4B is relatively modest, despite the carefully designed rewards. As a result, it is not clear whether the benefit over SFT alone sufficiently justifies the additional training stage and its complexity. Therefore, I think it would be interesting to see the contributions of SFT and RL reported separately for the 1.5B model as well. Without this decomposition, I cannot assess whether RL contributes more substantially in that setting or whether most of the improvement is already achieved through SFT.
  • I am also concerned that the improvement over the starting backbone is substantially smaller at 4B than at 1.5B. This raises questions about the method’s added value when applied to stronger embedding models.
Questions for authors
  • Could the authors clarify how much the RL stage improves CoFree-1.5B’s overall performance relative to its SFT-only checkpoint?
5 comments
AuthorsOpenReview ↗

Response to Reviewer yBBg (1/2): SFT and RL contributions

Thank you for recognizing the motivation, breadth of evaluation, and clarity of our work. We address the separate contributions of SFT and RL, the value of Stage 2, and the gains over the two initialization backbones below.

Separate contributions of SFT and RL (Weakness 1): I think it would be interesting to see the contributions of SFT and RL reported separately for the 1.5B model as well.

Question 1: Could the authors clarify how much the RL stage improves CoFree-1.5B’s overall performance relative to its SFT-only checkpoint?

Response: For CoFree-1.5B, RL improves Overall from 33.5 after SFT to 34.6, a gain of 1.1 points. SFT contributes 6.6 points to the total 7.7-point improvement over the starting backbone. At 4B, the corresponding gains are 1.9 points from SFT and 0.9 points from RL. This decomposition shows that SFT provides the larger share of the improvement at both sizes, while RL further improves retrieval beyond the SFT checkpoint.

Scores are Overall nDCG@10 (0–100), averaged across all 22 datasets: 10 from MTEB and 12 from BRIGHT. The 4B Full RL score is averaged over three runs initialized from the same Full SFT checkpoint.

Model sizeStarting backboneFull SFTFull RLSFT gain over backboneRL gain over SFT
1.5B26.933.534.6+6.6+1.1
4B38.039.940.8+1.9+0.9

Benefit relative to training complexity (Weakness 1): The additional gain from RL at 4B is relatively modest, despite the carefully designed rewards. As a result, it is not clear whether the benefit over SFT alone sufficiently justifies the additional training stage and its complexity.

Response: SFT provides the larger share of the retrieval improvement, while RL contributes additional gains at both model sizes, adding 1.1 Overall nDCG@10 points at 1.5B and 0.9 points at 4B. These are average improvements across 22 datasets covering both general retrieval and reasoning-intensive retrieval. The 4B result is additionally averaged over three RL runs initialized from the same SFT checkpoint.

The SFT-to-RL comparison also shows an improvement in the retrieval relevance of generated reasoning. At 1.5B, we assess whether reasoning helps distinguish relevant documents from hard negatives using BGE, Mixedbread, and Jina rerankers, all distinct from the training reward model.

CoFree-1.5B checkpointMean discrimination accuracy (%)
Full SFT65.60
RL without the reasoning reward64.47
Full RL72.97

Full RL improves discrimination accuracy by 7.37 percentage points over SFT. Removing the reasoning reward leaves accuracy below the SFT checkpoint, supporting its contribution to the relevance of generated reasoning.

The reward ablation further shows how the reasoning reward contributes to the content of generated reasoning. Three LLM judges assess intent preservation, reasonable and accurate expansion, and specific information contribution. Full RL receives higher scores than the variant without the reasoning reward under all three judges (Appendix D.6; scores range from 1 to 5).

CoFree-1.5B configurationQwen3.5-27BGLM-4-32BHunyuan-A13B
Without reasoning reward3.13.93.9
Full RL3.34.14.1

We believe that the current embedding reward design partly limits further retrieval improvements from RL. Future work will focus on refining this reward to improve the effectiveness of RL for embedding learning.

The additional retrieval gains and the improvement in reasoning relevance are why we include RL after SFT. The content evaluation further supports the reasoning reward used within this stage.

AuthorsOpenReview ↗

Response to Reviewer yBBg (2/2): Stronger backbone and deployment

Added value on the stronger backbone (Weakness 2): I am also concerned that the improvement over the starting backbone is substantially smaller at 4B than at 1.5B. This raises questions about the method’s added value when applied to stronger embedding models.

Response: We agree that the stronger Qwen3-Embedding-4B backbone leaves less room for further score improvements. CoFree raises this backbone’s Overall nDCG@10 from 38.0 to 40.8, providing a 2.8-point improvement on top of an already strong embedding model (Table 1). The resulting 4B model also outperforms CoFree-1.5B by 6.2 points.

Generated reasoning continues to provide an additional benefit with the stronger encoder. Adding CoFree reasoning to the final 4B encoder improves Overall by 2.1 points, close to the 2.3-point improvement at 1.5B (Appendix D.4 and the additional 4B inference ablation). This indicates that reasoning remains useful even when the underlying retrieval representations are already strong.

The framework’s practical value is further supported by our industrial application. We trained CoFree on industrial data and introduced it as an additional recall channel alongside existing Qwen3-based embedding methods in our e-commerce search system. The online A/B test yielded the following improvements (Section 4.2):

MetricImprovementType
Click-Through Rate (CTR)+0.02 pointsAbsolute
Conversion Rate (CVR)+0.12 pointsAbsolute
Orders+13.56%Relative
Gross Merchandise Volume (GMV)+8.15%Relative

Appendix F.2 further shows improvements in CTR, CVR, and orders across ten product categories. These results demonstrate that adding CoFree to an established retrieval system produces measurable benefits under real user traffic.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer yBBg,

I hope this message finds you well. With less than three days remaining in the discussion period, we would like to follow up on our response, particularly the separate SFT and RL results at both model sizes, the reasoning-quality analysis, and the discussion of the gains over the stronger backbone.

We would be grateful to hear whether these results and clarifications address your questions about the value of the RL stage and the improvements at 4B. If any questions remain, please let us know; we would be happy to discuss them further.

Thank you for your time and effort in reviewing our paper.

Best regards,
The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

Dear Area Chair,

Contribution and significance

CoFree identifies and measures reasoning collapse: embedding specialization can impair generation or produce fluent text that is unhelpful for retrieval. It addresses both failures through reference-guided SFT that restores generation while preserving embedding quality, followed by dual-reward RL that improves reasoning relevance and embedding discrimination. The work connects reasoning quality to retrieval utility through fixed-encoder controls, evaluation across 22 datasets, and an industrial online A/B test. This provides a concrete failure diagnosis and a practical approach to learning useful reasoning for embedding models.

Evidence strengthened during rebuttal

  • Attribution and reasoning sources (JjBD, XP3x, Knp4): Matched controls show CoFree-4B outperforming direct reranker distillation with reference guidance: 40.8 versus 36.8 Overall nDCG@10. Across five tasks and two fixed encoders, CoFree reasoning scores 41.1, versus 37.1 for teacher reasoning and 35.5 for Qwen3-4B reasoning. We clarified train–test overlap filtering and highlighted the added reasoning-source, distillation, and inference-ablation results in the revised manuscript.

  • RL benefits and variability (yBBg, XP3x): We clarified that RL adds 1.1 Overall points at 1.5B and 0.9 at 4B. All three 4B RL seeds score 40.7–40.9 versus the shared SFT starting point of 39.9. Independent evaluators also show improved relevance discrimination at 1.5B, from 65.6% to 73.0%.

  • Training, reasoning, and evaluation (xCfv): Our initial response compared instruction settings, separated the 4B gain into 0.7 points from encoder training and 2.1 from inference reasoning, and clarified the roles of task rewards and regularizers. In follow-up, we expanded frozen-backbone testing from five to all 22 datasets under matched dataset-specific instructions: +1.7 Overall points on frozen Qwen3 and +4.3 on frozen GTE. xCfv's latest recommendations are to emphasize the trained generator's contribution and add camera-ready evaluations using the backbone's released 8K/document-instruction configuration and BRIGHT's official excluded_ids filtering.

  • Baselines and retrieval scope (Knp4): Our initial response added ReasonIR, RaDeR, and ReasonEmbed comparisons, analyzed encoder-quality gaps, and showed how reference guidance mitigates degradation from continued fine-tuning. In follow-up, we clarified the goal of improving both general and reasoning-intensive retrieval. Added tests show CoFree reasoning improving all three frozen baseline encoders on both benchmarks, including +10.14 MTEB and +0.85 BRIGHT points for ReasonEmbed. We also explained the Search-R3 ranking difference through the different evaluation task sets.

Outcome of the discussion

The discussion clarified the source of CoFree's gains and the value of its generated reasoning. yBBg and xCfv both raised their recommendations from weak rejection to weak acceptance, with xCfv explicitly confirming that generated text adds retrieval value beyond task instructions. JjBD maintained their original clear-accept rating (4) after finding our responses convincing and their concerns substantially addressed.

The other two reviewers also acknowledged progress. XP3x confirmed that their main concerns were addressed and raised soundness and informativeness, while retaining an overall score of 2; we then incorporated and highlighted the requested results in the manuscript. Knp4 acknowledged that most concerns were addressed; we subsequently answered the remaining questions on retrieval scope and the Search-R3 comparison with additional experiments and clarifications. Neither reviewer has provided further follow-up after these replies.

We thank the reviewers for their constructive feedback and appreciate your consideration of the additional evidence in your assessment.

Best regards,
The authors of Paper 17884

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.