- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper investigates whether RLVR (primarily) improves embeddings in the same way that it improves reasoning, finding a null effect; the authors provide the explanation, based on further experiments, that the latent manifold is largely preserved, and that RLVR only changes local arrangements and movement on the manifold.
Critical strengths
I commend the authors for reporting a negative result. The null conclusion appears altogether robust.
Critical weaknesses
Another plausible and possibly more natural hypothesis is that RLVR affects only reasoning traces, and that only reasoning trace embeddings are affected. The training data for contrastive learning is normal text, and the evaluation of embeddings such as MTEB uses normal text. I did not see an experiment which refutes this hypothesis.
For example, when authors investigate differences between internal representations (RLVR or not RLVR), they almost exclusively use reasoning traces. Do the differences observed in internal representations (mostly in local arrangements) also occur with normal text? The appendix adds MMLU-Pro, where the local differences are already smaller, which the authors do not remark on. I do not understand why the authors did not test this hypothesis more strongly, this is why I have scored 1 on soundness, apologies if I am missing something.
Another question is whether a local overlap of say 0.5 should be judged as significant evidence of local re-organisation, rather than noise. Given the enormous amount of variation at play, it could be that sharing roughly half of your nearest neighbours is actually surprisingly high.
Questions for authors
Do the differences observed in internal representations (mostly in local arrangements) also occur with normal text?
What evidence can the authors give that the difference in overlap detected is significant?
Thank you for the questions. We address your concern below.
Q1. Do differences appear outside reasoning traces?
Thank you for highlighting input-domain dependence. Smaller local differences on MMLU-Pro are compatible with Manifold Realignment, which does not require equal effects across input distributions.
The starting backbones already show incomplete neighborhood agreement on MMLU-Pro. For DS–ProRL, k-NN overlap at k = 10 is 0.5044, comparable to 0.4841 on CoT. After embedding adaptation, MMLU-Pro Linear CKA increases from 0.8477 to 0.9757, while k-NN overlap increases to 0.7482 (Figs. 8–11). Thus, the higher endpoint overlap accompanies partial realignment, rather than indicating identical starting representations. On CoT, CKA also increases, while local overlap remains similar or decreases.
Our functional analysis additionally includes AG News and ANLI, and downstream evaluation covers the MTEB Multilingual (which include 131 tasks), MTEB Code, and BRIGHT. These support the preservation of the tested readouts and the lack of a consistent aggregate embedding advantage.
We will explicitly discuss the domain-dependent effects and clarify the MMLU-Pro input construction. Stronger effects on reasoning traces can coexist with our structural findings, although the current evidence does not uniquely establish why the downstream advantage is small.
Q2. Is overlap around 0.5 meaningful or noise?
An overlap around 0.5 does indicate substantial preservation, consistent with our finding that RLVR preserves considerable structure while changing some local relationships.
Our interpretation rests on several main observations. First, the pattern is consistent across three seeds and k = 5, 10, and 50. At k = 10, DS–ProRL achieves 0.484 ± 0.065, compared with 0.068 ± 0.027 for the SFT pair, with the same ordering at the other neighborhood sizes (Table 3). Second, the comparison uses identical input tokens, and the metric is invariant to rotations and uniform scaling (Appendix B.2.2), so these neighborhood differences cannot be explained solely by a change of coordinate basis.
Third, local differences persist alongside global realignment. After embedding adaptation, DS–ProRL Linear CKA increases from 0.8946 to 0.9498, while k-NN overlap with k = 10 remains approximately unchanged at 0.474 ± 0.069 (Figs. 8–9; Table 3). Together with strong probe transfer and similar embedding performance, we show the central observation where global and functional agreement can coexist with nonidentical local neighborhoods. We interpret the overlap through this consistent multilevel pattern intended in the design of HRSA, rather than treating 0.5 alone as a significance threshold.
We hope these clarifications address the main concerns, and we would be grateful for a re-evaluation.