- Rating
- 2
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether a purely supervised retriever can match Q-RAG on long-context, multi-step retrieval. The authors attribute the shortcomings of supervised methods such as MDR and Beam Retriever to three factors: negatives that are too easy, later hops being trained on incorrect predicted prefixes, and failure to use a chunk’s position relative to what has already been retrieved. S2LMR addresses these issues with a single shared bi-encoder, a per-hop InfoNCE loss over all chunks from the same context and reference prefixes during training.
Critical strengths
The method is simple and easy to implement. When chunk-level supervision is available, removing the value network, the target network and the TD targets is a real practical simplification.
The analysis of negatives in Section 3.1 is the most interesting part for me. Table 6, Figures 3, 7 show quite convincingly that MDR-style in-batch training ends up separating the wrong distribution.
The evaluation covers both long synthetic tasks and standard open-domain multi-hop QA.
Critical weaknesses
-
The positional encoding is not new. Q-RAG also recomputes positions after each hop. The real novelty is replacing TD learning with teacher-forced InfoNCE over the whole context.
-
The training-time and stability comparison is unreliable. Table 4 reports ~54h for Q-RAG on QA3, but the Q-RAG paper says at most 12 hours on one A100, with curves flattening in about two hours. The Q-RAG run in Figure 6a also only gets to 0.876 on QA3, while Figure 4 in this paper shows Q-RAG at about 0.98. So the 24x speedup and the whole of Section 4.3 seem to be based on a Q-RAG run that wasn't trained properly.
-
The reproduced Q-RAG numbers are lower than the original and change the conclusions. On RULER QA, HotpotQA, and MuSiQue, the original Q-RAG numbers are higher and beat S2LMR at every length.
-
The method needs fine-grained labeling. It requires gold chunks in a fixed hop order and K at test time. HotpotQA has no natural order, and document order is only a convention. Q-RAG only checks whether all support facts were found, could use an answer-based reward, does not penalize other valid paths, and can use Q-values to stop. None of this works for S2LMR, and the paper should discuss it.
-
Minor: CCR and label smoothing are used but never defined. Delta is 200 for QA3 and 20 for other tasks, with no explanation. Table 8 has identical numbers to three decimals for all N. Chunk-axis encoding is disabled on RULER, HotpotQA, and MuSiQue, but this is not stated in the main text.
Questions for authors
- How does S2LMR behave when K is not known at test time, for example with a larger fixed budget or with a learned stopping rule?
- Can S2LMR be used when only answer-level supervision is available, for example exact match or an LLM judge on the final answer? A small experiment, for example on HotpotQA without supporting-fact labels, would help a lot.
- How are CCR and label smoothing defined, and how do the results change without them?
- Q-RAG reports out-of-distribution results on MuSiQue after training only on HotpotQA. How does S2LMR do in the same out-of-distribution setting?
5 comments
Response to Reviewer WkUx
S²LMR is better under both protocols. Even with 4 steps and no stop, it reaches 0.77 Fact EM and 0.62/0.47 Ans F1/EM, still above Q-RAG with the stop. We will state all settings and protocols in the paper.
W4: Labeling requirements.
This is a valuable point, and we will add this discussion to the paper. None of these requirements is intrinsic to S²LMR:
- Known . S²LMR can run with a fixed budget, or with a learned stopping rule on its own retrieval scores. The latter plays the same role as Q-RAG's Q-value stopping (see reply for Q1).
- Fixed hop order. The per-hop loss can take a soft minimum over all orderings of the gold set. Training then only requires the full set to be retrieved, and no ordering of it is penalized. Without any order labels, this matches the fixed-order model in distribution (Fact EM: HotpotQA 0.90 vs. 0.90; MuSiQue 0.69 vs. 0.67) and transfers better across datasets (see reply for Q4).
- Supporting-fact labels. These can be replaced by pseudo-labels derived from the answer alone (see reply for Q2).
We also note that the results Q-RAG reports are likewise trained with rewards computed from gold supporting facts.
W5: Minor issues.
We will fix all four.
(i) CCR and label smoothing. Both regularize the per-hop InfoNCE:
where is label smoothing over and the entropy term is a confidence penalty (Pereyra et al., 2017). Removing either changes Fact EM only marginally (Fig. 8–9).
(ii) . Performance is stable for on QA3 (0.96–0.99 Fact EM) and drops at (0.94–0.95). QA3 needs this finer positional resolution because its gold facts are tightly clustered (Table 7).
(iii) Table 8. The identical values are expected: a position-agnostic encoder scores text only, and both the gold and the strongest distractor are bAbI facts present at every , while the added PG-19 filler never outranks them. The failure is therefore semantic, not caused by pool size.
(iv) Chunk-axis PE. We will state in §4.1 that it is disabled on RULER, HotpotQA, and MuSiQue, where chunk order carries no cross-hop signal (shuffled paragraphs or randomly inserted needles).
Q1: Unknown at test time.
Thanks for this great question! We study this on MuSiQue, where questions need 2–4 paragraphs and is not given. Without changing the trained retriever, we compare three rules:
- larger fixed budgets;
- a training-free stop that halts when the next pick's score falls below a threshold;
- a lightweight learned stop: a logistic regression on S²LMR's own per-hop scores (pick score, top-1/top-2 margin, score drop) that predicts whether the evidence is already complete.
Thresholds are calibrated on held-out data (2-fold on dev). Results on the full dev set:
| Retrieval rule | Fact EM | Avg. #picks |
|---|---|---|
| Known | 0.67 | 2.65 |
| Fixed / 3 / 4 | 0.39 / 0.65 / 0.77 | 2 / 3 / 4 |
| Score threshold | 0.63 | 2.60 |
| Learned stop | 0.65 | 2.70 |
A larger fixed budget is a safe default. With , evidence recall rises to 0.77 and answer quality also improves (QwQ-32B Ans F1/EM 0.62/0.47, vs. 0.56/0.42 with known ), as the reader tolerates a few extra paragraphs. When a compact evidence set is preferred, the learned stop recovers most of the known- recall (0.65 vs. 0.67) at nearly the same cost (2.70 vs. 2.65 picks), without retraining the encoder.
Q2: Answer-level supervision.
Great question! On HotpotQA, we discard all supporting-fact annotations and derive training targets from the answer alone. An LLM (Qwen3-8B) reads the question, the gold answer and the context paragraphs, and names the two paragraphs needed, in reasoning order; this order is used for teacher forcing. Supporting-fact labels are used only for evaluation. The pseudo-labels are imperfect: both paragraphs are correct for 77% of questions. Even so, with the same training recipe, S²LMR trained on them reaches 0.88 Fact EM on the full dev set, close to 0.90 with gold labels and well above the zero-shot 0.65. Answer-derived labels therefore suffice to train S²LMR.
Q3: Definition of CCR and label smoothing.
See reply for W5.
Response to Reviewer WkUx
Q4: Out-of-distribution transfer.
We follow Q-RAG's protocol (up to 4 steps on MuSiQue, stopping once all gold paragraphs are found; 2 steps on HotpotQA). S²LMR matches Q-RAG on HotpotQA → MuSiQue and is ahead on MuSiQue → HotpotQA. With the order-agnostic objective from W4, it is ahead in both directions:
| Fact EM | HotpotQA → MuSiQue | MuSiQue → HotpotQA |
|---|---|---|
| Q-RAG | 0.55 | 0.68 |
| S²LMR | 0.54 | 0.76 |
| S²LMR, order-agnostic | 0.60 | 0.77 |
We will add these results to the revision.
Response to Reviewer WkUx
We sincerely thank the reviewer for an exceptionally careful, detailed and helpful review.
W1: Positional encoding.
Yes, we agree with the reviewer's comments on the positional encoding and the teacher-forced InfoNCE. As we wrote in the paper, the chunk-axis encoding, including its per-hop recomputation, is inherited from Q-RAG. S²LMR's only modification is to compute positions from the gold prefix during training.
One important part of our contribution is showing why the key components such as positional encoding and teacher-forced InfoNCE are needed and that they suffice: supervised retrievers fall behind RL not because supervision is too weak, but because their training is misaligned with inference. With these misalignments removed, a plain supervised bi-encoder matches or outperforms Q-RAG from 4K to 10M tokens, without value networks or TD updates.
W2: Training time and stability.
We thank the reviewer for this careful check.
Training time. The 54.4 h in Table 4 is the wall time of the full schedule of Q-RAG's released configuration on one A6000. We will revise the speedup claim to: "On BabiLong QA3, S²LMR reaches its peak performance in 1.4 GPU hours and completes training in 2.25, compared with 3 and 54.4 for Q-RAG."
"0.876 vs. 0.98." The gap comes from the evaluation protocol. Fig. 4 follows the Q-RAG paper: a budget of T=6 retrieval steps, with credit whenever all gold facts are retrieved. Fig. 6a evaluates both methods under the same stricter protocol, with T=K=3 steps that must all be correct. We will state both protocols explicitly.
Other parts of Section 4.3. The robustness results (Fig. 5) do not depend on this run: they come from separate learning-rate and batch-size sweeps, and the specific budget (T) for each run is reported.
W3: Reproduced Q-RAG numbers.
We thank the reviewer for pointing this out! In the revision, we report Q-RAG's original numbers together with results of the authors' released checkpoints. Under matched settings, our conclusions hold on all three benchmarks.
RULER QA. In submission we use the authors' released Q-RAG checkpoint, without retraining.
EM at 4K / 16K / 32K / 128K / 1M:
The two settings agree: on MH, S²LMR matches or outperforms Q-RAG at every length; on SH, it is better at 4K and behind at longer contexts.
HotpotQA and MuSiQue. Our submission retrained Q-RAG with the official repository, whose configuration differs from that of the released checkpoints. We now compare against Q-RAG's original numbers and its released checkpoints.