- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies Recursive Language Models (RLMs) and points out that selection among candidate interaction programs under uncertainty rather than recursion itself is the key to success in long-context reasoning; programmatic interaction contributes majorly for the success of RLM. Based on this, the paper proposes Self-Reflective Program Search for Long Context Method (SRLM), which samples different trajectories, conducts majority voting, and selects the one with highest confidence and longer reasoning length. On several benchmarks and two different models, the proposed method outperforms several baselines.
Critical strengths
-
This paper studies an interesting question on recursive language models, which is timely in today's LLM community with long-context tasks. The conclusion that the gain of RLM mainly comes from programmic context interaction is quite insightful.
-
The proposed method achieves significant performance improvement compared to RLM and other baselines.
Critical weaknesses
- I think the key concern about this paper is that the design of SRLM feels somewhat ad-hoc. Compared to RLM, SRLM is different only with majority voting and a heuristic tie-breaker (indicated from Fig. 1, which somewhat contradicts with line 169-170; see questions for details) that uses a product of LLM-generated confidence score and response length. Judging from line 186, the verbalized confidence is a sum across steps instead of mean, which is negatively correlated with reasoning length.
Among the three components:
a) Majority voting is a common technique for test-time scaling that improves performance at the cost of multiple times of token consumption. While wall-clock time consumption is reported in Fig. 3, multiple rollouts can be completed in parallel (especially for GPT-5), giving majority voting a hidden and unfair advantage.
b) LLMs are known to be overconfident [1] as a judge; thus, it is not sure whether the confidence score generated by LLM is reliable or not. While the paper reports the relation between confidence, length and accuracy in Fig. 5 and 13, the distribution of confidence reported by LLM is not known.
c) Reasoning length also does not necessarily indicate more intricate reasoning, especially for very strong models such as GPT-6 Astra which can hide CoT [2] or weaker models which can fall into repeat loops during reasoning [3].
- The central question raised in line 103-104 feels somewhat unconvincing. Fig. 4 shows significant improvement of adding recursiveness (i.e. sub-call) to both RLM and SRLM. Also, "The real bottleneck is how we select among candidate interaction programs under uncertainty" seems to be leveraging the unfair advantages of more test-time resources (multiple trials) that RLM has no access to.
References
[1] https://arxiv.org/pdf/2508.06225
Questions for authors
-
Since after majority voting for self-consistency, as suggested by Fig. 1, all remaining responses have the most frequent answer, the other two factors (reasoning trace length and verbalized confidence) are only tie-breakers among these answers. The problem is: how often does this tie-breaker mechanism come into effect? Also, the definitions in line 169-170 seems to indicate that only one plurality answer is selected. In this case, reasoning trace length and verbalized confidence should not have any effect on accuracy since the candidate answer is unique. Could the authors explain this?
-
Why the authors would use LLM judges for all three benchmarks tested in this paper (BrowseComp+, OOLONG-synth and LongBench-v2)? LLM judges could introduce additional hallucination and inaccuracies in scoring, while both OOLONG-synth and LongBench-v2 has verifiable answers that does not require LLM judges (especially LongBench-v2, which are multiple choice questions). Is there any empirical evidence on the accuracy of LLM judge?
6 comments
Response to reviewer yTVJ (2)
The central question raised in line 103-104 feels somewhat unconvincing. Fig. 4 shows significant improvement of adding recursiveness (i.e. sub-call) to both RLM and SRLM. Also, "The real bottleneck is how we select among candidate interaction programs under uncertainty" seems to be leveraging the unfair advantages of more test-time resources (multiple trials) that RLM has no access to.
We are not fully sure we have understood the reviewer's comment, we appreciate if the reviewer can further clarify on this and let use know if any further clarification from us could help. As it can be observed from Fig 4, the gain from recursion differs significantly between different domains and this gain is more significant in search-oriented domains like CodeQA and Structured Data QA compared to more semantic domains like single or multi-doc QA. What we wanted to share as insight in this paper was that the choice of inference-time scaling approach plays an important role in these frameworks and there could be different ways to spend additional inference compute for this purpose, we can spend additional inference compute through recursion sub-calls (sub-agent setting) or we can spend that compute through simpler ways like best of N and self-reflection, similar to the step-wise uncertainty-based self-reflection we have in SRLM. We wanted to share the insight that even simpler inference-time scaling approaches could be as effective or even better than the recursion in this long context interaction setting and recursion is not as fundamental as it’s claimed for long context in “recursive” language model (RLM).
Why the authors would use LLM judges for all three benchmarks tested in this paper (BrowseComp+, OOLONG-synth and LongBench-v2)? LLM judges could introduce additional hallucination and inaccuracies in scoring, while both OOLONG-synth and LongBench-v2 has verifiable answers that does not require LLM judges (especially LongBench-v2, which are multiple choice questions). Is there any empirical evidence on the accuracy of LLM judge?
We use an LLM judge to make scoring robust to how answers are formatted. Rule-based answer extraction can mark a correct answer wrong just because it is phrased differently, and this adds wrong penalties to evaluations. It’s important to note that the judge’s task is quite simple here: it only checks whether the LLM final answer matches the ground truth. So it adds very little judgment of its own while removing errors caused by formatting. In response to reviewer's comment, we also measured agreement between the judge and human scoring among 50 random samples and found complete agreement.
We hope that our rebuttal address the reviewer's concerns, and if so, they would consider updating their score. We'd be more than happy to engage in further discussions
Thank you for your response, and thank you for explaining how the three components of SRLM works together at different levels. Here are my follow-up questions:
-
Regarding heuristic designs: I agree that heuristic methods are not necessarily bad. My point is that the metrics (length and self-generated confidence score) chosen by this paper is found to be potentially problematic as stated in my reference papers, and there is no rationale on why the way that the authors combine these metrics at each step (by using product) is the best way. Also, my question " the distribution of confidence reported by LLM is not known" is not answered: this is an empirical validation to my concern "LLMs are overconfident to be the judge" on the confidence score.
-
Regarding the statement in your response "we can spend that compute through simpler ways like best-of-N and self-reflection": this is not verified in the experiment result; currently, the paper misses key baselines which applies those "simpler ways". While Fig. 5 provides some ablation, there is no formal results with baseline using the same number of candidates. There is also no report on the total token / function call consumption of each baseline (Fig. 3 reports the time consumption, but it does not reflect the actual usage of tokens as trajectories can be produced in parallel to reduce time).
-
Regarding line 103-104: The question asked in your paper is "Is recursion itself the key ingredient for long-context reasoning, or is the real bottleneck how we select among candidate interaction programs under uncertainty"? My point is:
a) Since adding recursiveness to both RLM and SRLM both improves performance, I think it can be called a key ingredient for long-context reasoning; it's just that it works better with SRLM. Maybe recursiveness is not the sole or most important ingredient, but it still can be listed as a key ingredient.
b) to select some candidate under uncertainty, one must have multiple candidates to choose from. However, the most important baseline, RLM, does not have such candidates of parallel trajectories to begin with. Thus, I think "selecting candidates as a bottleneck" (if we are allowed to do parallel runs) has no contradiction with the statement "recursion is the key ingredient for long-context reasoning (on the RLM's setting where one is not allowed to do parallel runs)". They should not be connected with the word "or".
c) In your response to the other reviewers, you mention "each RLM score is the average over 8 samples, so RLM and SRLM draw on the same number of trajectories." This comparison is unfair - 8 samples for RLM is essentially 8 independent trials while only 1 trial for SRLM. A fair comparison would be using majority@8 (especially for multiple choice benchmarks) or any other "simpler ways" to aggregate the result (e.g. self-reflection). The performance of SRLM against majority@8 (+self consist.) seems relatively limited in Fig. 5.
Response to reviewer yTVJ (1)
Thank you for dedicating your time and expertise to review our submission. Please find our responses below.
Thank you for recognizing the strengths and novelty of our work.
Thank you for the comment. We would like to highlight that most of the uncertainty quantification metrics in LLM literature are in fact heuristic (including self-consistency [1], verbalized confidence [2], and trace-length-based uncertainty [3], or their integrations) and our metric is also built motivated by these prior works. The contribution of this paper is not that we have proposed a new and better uncertainty quantification metric, but that programmatic interaction is a new way of handling long context through inference-time compute, and the choice of inference-time scaling approach plays an important role here. There could be different ways to spend additional inference compute for this purpose, we can spend additional inference compute through recursion sub-calls (sub-agent setting) or we can spend that compute through simpler ways like best of N and self-reflection, similar to the step-wise uncertainty-based self-reflection we have in SRLM. We wanted to share the insight that even simpler inference-time scaling approaches could be as effective or even better than the recursion in this long context interaction setting, which has been proposed as the key direction for long context handling based on the claim of "recursive" language models.
References:
[1] Wang, X. et al. "Self-Consistency Improves Chain of Thought Reasoning in Language Models.", 2022
[2] Xiong, M. et al. "Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs.", 2023
[3] Devic, S et al. "Trace Length is a Simple Uncertainty Signal in Reasoning Models.", 2025
Thank you for raising thoughtful question. We would like to clarify that in SRLM, the three signals work at different levels. Verbalized confidence and reasoning length are applied per step: they guide which intermediate steps are kept as each context-interaction trajectory is built. Self-consistency is applied over the full sequence of steps, based on the final answers of the completed trajectories. They are not a re-ranking inside a fixed answer group, so the reviewer's statement that “reasoning trace length and verbalized confidence should not have any effect on accuracy” is not true as with these signals, the candidate pool itself can be different. This also explains our component ablation results in Fig. 5 where we show combination of all three metrics is helpful to performance. We agree that our current writing could be clearer and thank reviewer for bringing this to our attention. We have now revised Section 2.2 to state where each signal acts, and we added a short algorithm description to make the procedure clearer (edits highlighted with blue).