acceptodds
Under review as a conference paper at ICLR 2027

Informed Recursion: Structural Awareness for Recursive Language Agents

Abstract

Recursive Language Models allow language models to undertake context lengths beyond their designed context windows by treating long prompts as external environments and recursively querying themselves. However, a closer look at this performance reveals a large drop in accuracy on tasks requiring exact numerical counts relative to comparison and existential tasks. We trace this gap to a failure mode in which ordinary classification errors near ambiguous category boundaries are amplified by exact-count and combinatorial aggregation. We introduce Informed Recursion, a single-trajectory approach with two lightweight mechanisms: Query-Structure Routing, which selects recursion depth based on the structure of the query, and Targeted Verification, which re-checks intermediate decisions that can affect the final aggregation. Across our evaluations, Informed Recursion improves RLM performance by adapting recursion depth to query structure and selectively verifying intermediate decisions. Doing so avoids the cost of applying deeper recursion or multiple trajectories to every query. Our analysis indicates that many RLM failures arise from local classification errors that become amplified when their outputs are aggregated, rather than from incomplete access to the context.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Recursive Language Models, Language Model Agents, Long-Context Reasoning, Recursive Inference, Inference-Time Compute, Task Decomposition, Agent Orchestration

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_uq6BOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Informed Recursion, which selects RLM recursion depth based on query structure and selectively verifies intermediate decisions. It aims to reduce errors on exact-count and pairwise aggregation tasks, and evaluates the approach on several long-context and reasoning benchmarks.

Critical strengths

The method is simple and practical, combining adaptive recursion depth with targeted verification. The improvements on OOLONG-Pairs and the depth ablation results demonstrate its potential effectiveness.

Critical weaknesses
  1. Limited significance and generalizability: My main concern is that the paper addresses a rather narrow failure mode with relatively straightforward heuristics, without providing much generalizable insight into recursive reasoning. The observation that classification errors can be amplified during aggregation is fairly intuitive, and the proposed routing and verification strategies are not particularly novel. More importantly, the authors themselves acknowledge that the method does not consistently generalize beyond aggregation-sensitive tasks, as also reflected in the mixed results on LongCoT-mini. While the improvements on OOLONG are encouraging, I find it difficult to identify a broader methodological principle or insight that can be applied beyond this specific setting. This significantly limits the contribution of the paper.

  2. Insufficient support. The failure analysis lacks direct evidence that classification errors are the main cause of RLM failures. In Table 4, the reported LongCoT-mini overall accuracy (44.90%) conflicts with the five domain accuracies, which imply 46.00% over 50 equally weighted tasks. This needs correction or a clear explanation.

Questions for authors
  1. How was the 44.90% overall result in Table 4 computed?
  2. Can the authors provide a more in-depth error analysis to demonstrate that the observed failures are indeed caused by the identified issues, and that the proposed method effectively addresses these specific failure modes?
  3. Can the authors provide broader experiments to clarify when the proposed mechanisms are effective and how well they generalize beyond aggregation-sensitive tasks?
3 comments
AuthorsOpenReview ↗

Response to Questions on Results, Error Analysis, and Generalization

Thank you for the feedback. We have addressed each question below and have incorporated the relevant corrections and analyses into the revised manuscript.

1)During our final evaluation passes, we identified and corrected a scoring issue affecting one Longcot result. And although the domain-level accuracies were updated, the overall accuracy was mistakenly left at its previous value of 44.90%. The correct value is 46.00% (23/50 correct). We apologise for this oversight. Table 4 has been corrected in the revised manuscript.

2)We revisited the original GPT-5.6 Luna trajectories and performed an item-level analysis of an OOLONG-Pairs query.

Evidence for the failure mechanism: We recovered all 1,585 intermediate classifications from the fixed Depth-2 and Full Informed Recursion executions and compared them against reference labels.

Fixed Depth-2 made 149 classification errors (90.60% accuracy), while Full Informed Recursion made 97 errors (93.88% accuracy). Independently reaggregating the recorded classifications reproduced both final predicted pair sets exactly.

We then performed controlled counterfactual corrections on the Full trajectory. Correcting one incorrect classification eliminated 101 false-positive pairs, increasing F1 from 75.73% to 76.49%. Correcting two eliminated 202 false-positive pairs, increasing F1 to 77.26%. Additionally, 25 of the 97 incorrect classifications individually affected the predicted pair set when corrected in isolation.

These results directly demonstrate how intermediate classification errors can propagate through membership conditions into larger aggregation errors.

Evidence for the proposed approach: On the same query, Full IR achieved 75.73% pair F1, compared with 67.44% for fixed Depth-2, along with the reduction from 149 to 97 classification errors.

This provides task-level evidence that the combined configuration can improve both intermediate classification accuracy and final outputs. The analysis establishes the propagation mechanism in this case, rather than claiming that it is the sole cause of errors across the benchmark. We have Added this analysis to Section 3.3.

3)We have noted that the effectiveness of Informed Recursion must be distinguished from its generalizability.

Our evaluation includes long-context aggregation, repository comprehension, multi-domain reasoning, and held-out multi-document retrieval. The results show a task-dependent pattern:

1)OOLONG-Pairs: 78.31% F1 for Full Informed Recursion versus 70.53% for fixed Depth-2. 2) OOLONG: 66.50% acc vs 65.97% for fixed Depth-2. 3) CodeQA: 66.00% acc vs 64.00% for Depth-1 and 62.00% for Depth-2. 4) Longcot-mini: 46.00% for Full IR vs 48.00% for Depth-1. 5) BrowseComp-Plus: 89.33% acc vs 82.00% for the evaluated RLM baseline.

These findings indicate that the strongest observed improvement occurs on OOLONG-Pairs. Results on other reasoning tasks are more variable, as shown by LongCoT-mini. Our approach does not improve performance across all reasoning tasks.

This distinction has been clarified in Section 6.4 and the conclusion now emphasizes the aggregation-sensitive setting rather than claiming a general improvement to recursive reasoning.

Thank you for the detailed response and additional analysis. I appreciate the authors' efforts to address my concerns. However, my overall assessment of the paper remains unchanged, particularly regarding its limited novelty and generalizability. Therefore, I will maintain my original rating.

Thank you for considering our response and the additional analysis.

We understand that your concerns regarding novelty and generalizability remain. However, we would like to clarify that our contribution is not simply the combination of routing and verification.

Our main contribution is identifying and examining a specific failure mechanism in recursive language agents, where errors in intermediate classifications can lead to much larger errors during aggregation. The additional item-level analysis provides direct evidence of this mechanism. We recovered 1,585 classifications, reconstructed the final pair sets from those classifications, and showed that correcting just two incorrect labels eliminated 202 false-positive pairs. We believe this provides a more concrete understanding of where recursive reasoning fails and where interventions can be useful.

Regarding generalizability, we agree that our method does not improve every reasoning task. However, the evaluation also includes a held-out BrowseComp-Plus setting, where accuracy improved from 82.00% to 89.33%, and CodeQA, where the Full system achieved 66.00% compared with 64.00% for the best fixed-depth baseline. We have reported the negative LongCot-mini results as well, rather than claiming universal improvements.

We recognize that these results may not fully resolve your concerns, but we believe the revised analysis and the results outside OOLONG deserve consideration when assessing the paper's contribution.

Reviewer_YZfgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that Recursive Language Models (RLMs) do worse on exact-count and pairwise-aggregation queries because small classification errors near category boundaries get amplified by aggregation, not because context coverage is incomplete. It proposes Informed Recursion, a prompt-level single-trajectory method with two parts: Query-Structure Routing (an LLM call that assigns recursion depth 0/1/2 from the query) and Targeted Verification (a re-check of selected intermediate decisions). It is evaluated with GPT-5.6-Luna on OOLONG 131K (50 tasks), OOLONG-Pairs 64K (20 queries), LongBench-v2 CodeQA (50), LongCoT-mini (50), and held-out BrowseComp-Plus (150), against RLM Depth-0/1/2 and SRLM (K=5).

Critical strengths
  1. A concrete, checkable framing of aggregation sensitivity. In pair-construction tasks, one misclassified user enters every pair that user participates in, so the error scales with the size of the other set (Sec. 3.3; Task 15, where two corrected decisions remove 1,558 spurious pairs). The paper does not establish this as the dominant cause, but it is a specific hypothesis about where long-context agents lose accuracy, and it points to cheap interventions at particular decision points instead of more samples.

  2. A large gain on the target benchmark at modest cost, reported candidly. On OOLONG-Pairs the full system reaches 78.31 F1 versus 70.53 for the best fixed depth (Depth-2), at $0.0869 versus $0.0724 per task, and far below SRLM's $0.27–0.32. The paper also gives mean ± std costs, per-cohort ablations, the exact prompts, and the negative LongCoT-mini result, and it limits its conclusion to aggregation-sensitive tasks. This makes the results easy to inspect, which is how the problems below could be found.

Critical weaknesses
  1. The diagnosis is untested, parts of the paper's own evidence contradict it, and the method as implemented does not clearly operationalize it.

    • The audit (Table 1) only rules out two alternatives: lines read and duplicate calls. It reports no classification accuracy against gold, no split of final error into classification versus aggregation error, and one anecdote (Task 15). Loading 100% of lines into the REPL does not show every record was classified by a sub-call. Sub-agents also run at Low reasoning effort, an untested alternative cause of the "boundary errors."
    • The motivating gap (exact-count worse than existential/comparison) is never shown. There is no OOLONG breakdown by task type. The Sec. 3.2–3.3 formulas are bookkeeping for cross-products and apply equally to both sets. "Existential" means an ∃ predicate in Sec. 3.2 but a monotone-predicate cohort in Table 3.
    • Table 3 runs against the thesis. Verification-only helps the existential cohort (79.05 vs 70.94 for Depth-1) and hurts the hybrid cohort (55.76 vs 64.57), the opposite of what the mechanism predicts. Depth-2 also does not help the hybrid cohort (63.38 vs 64.57); its overall gain comes from the existential cohort. This conflicts with "aggregation-heavy queries need deeper recursion."
    • The router prompt (App. A.1) never mentions counting or aggregation. Its categories are lookup / multi-hop / multi-predicate / open-ended / cross-file, while Sec. 4.1 and Fig. 2 describe routing by exact-count structure. For verification, there is no rule for choosing "critical" decisions, no count of decisions verified, and no cost. It is the same model checking its own outputs, with no measurement of how many flips are correct.
  2. The empirical evidence is too noisy and internally inconsistent to support the performance claims.

    • Samples are 20 queries (10 per cohort in Table 3) and 50 elsewhere, with no confidence intervals or seeds. The motivation for routing (Sec. 3.4, Depth-2 hurts CodeQA, 62 vs 64) is one task. Full vs Depth-1 on CodeQA (66 vs 64) is one task. Full vs Depth-2 on OOLONG is 0.53 points.
    • The OOLONG baseline is anomalous. Depth-1 scores 42.00%, below Depth-0 (56.00%) and the truncated base LLM (50.00%), while Depth-2 scores 65.97%. The large OOLONG gain in Fig. 1 is largely against this weak baseline, and Fig. 4 omits OOLONG, where depth is non-monotonic.
    • The ablation numbers do not fit together. If the router sends OOLONG to Depth-2, routing-only (57.00) should match Depth-2 (65.97). Verification-only gives essentially no gain on any benchmark (67.40 vs 67.76; 64.00 vs 64.00; 56.33 vs 56.00), and routing-only hurts CodeQA (58 vs 64). Yet adding verification to routing gives +6.0 / +8.0 / +9.5. Either router assignments vary or run-to-run noise is around 9 points; both undermine the comparisons. The full system is also cheaper than routing-only on Pairs and CodeQA despite extra verification calls, with std roughly equal to the mean.
    • The baselines are weak or missing. SRLM at K=5 scores below plain RLM (63.51 vs 67.76 on Pairs; 60 vs 64 on CodeQA), so beating it says little about single- versus multi-trajectory. There is no compute-matched baseline (e.g., higher sub-call reasoning effort, or verifying all decisions). The cost-saving claim versus fixed depth is unsupported: the full system costs more than Depth-2 on Pairs.
    • Reporting has errors. Fig. 1 claims gains at "nearly every context length," but on Pairs the method is below Depth-1 at 128K and 256K (read from the figure) and about equal at 64K on single-doc OOLONG. In Table 4 the overall 44.90% does not match the per-domain values (their mean is 46.0%); 44.90% equals 22/49, which suggests a dropped task.
    • The primary results are in-sample. The prompts contain task-type text ("pair/relation tasks," "code/retrieval tasks"), and individual tasks were inspected. Only BrowseComp is held out, and there the baseline depth is not stated and the router's depth-2 criterion ("open-ended, multi-document exploratory search") closely matches the benchmark.
Questions for authors
  1. Can you report intermediate classification accuracy against gold and decompose final error into classification versus aggregation error? Without this, what supports the claim over alternatives such as Low-effort sub-calls or selection errors? Can you show the claimed exact-count vs existential gap on OOLONG by task type?
  2. Why does verification-only help the existential cohort and hurt the hybrid cohort in Table 3, if it targets amplified aggregation errors?
  3. What depth does the router assign on each benchmark, how accurate is it against an oracle, and why is routing-only 57.00 on OOLONG when Depth-2 is 65.97? Does the router cost appear in Table 2?
  4. How are critical decisions selected, how many are verified per task, how often do flips turn wrong into right versus right into wrong, and what is the added cost?
  5. Can you give confidence intervals or multiple seeds, plus per-task win/loss counts, for the main comparisons? Please also explain the 44.90% LongCoT-mini figure and the Depth-1 result of 42.00% on OOLONG.
1 comment
AuthorsOpenReview ↗

Response to Technical Questions and Additional Experimental Analysis

Thank you for the detailed questions. We revisited the original GPT-5.6-Luna trajectories and added the following analyses to the revised paper.

  1. Classification errors and task-type gap

For one audited OOLONG-Pairs query, we compared 1,585 intermediate classifications with the reference labels. Fixed Depth-2 made 149 errors (90.60% accuracy), while Full Informed Recursion made 97 (93.88%). Reaggregating the recorded labels reproduced both final pair sets exactly, so we found no separate pair-construction error in this query.

In a post-hoc analysis of the Full trajectory, correcting one wrong label removed 101 false-positive pairs, while correcting two removed 202. This shows how a few classification errors can affect many output pairs. These were controlled corrections, not actual verifier interventions. Low-effort subcalls or selection errors could still contribute to the original mistakes.

On OOLONG 131K, fixed Depth-2 averaged 8.21% on 12 exact-count questions (0 exact matches), versus 82.76% on 29 frequency-comparison questions (24 fully correct). The 74.55-point gap uses the benchmark's partial-credit score for numerical answers. There was no separate existential category in this OOLONG breakdown. We clarified this in Section 3.

  1. Verification-only results

We agree that verification alone is not consistently helpful. It raises existential F1 from 70.94% to 79.05%, but lowers hybrid F1 from 64.57% to 55.76% (Table 3).

Incorrect verification changes may be especially costly for hybrid tasks involving exact-count conditions, although these results do not prove that explanation.

The Full system achieves 85.46% existential and 71.15% hybrid F1. We clarified this distinction in Section 6.3.

  1. Routing assignments and cost

The original router assigned depths 0/1/2 as follows:

  • OOLONG (50): 38/0/12; agreement with best observed fixed depth: 76%.
  • OOLONG-Pairs (20): 0/0/20; agreement: 50%.
  • CodeQA (50): 0/50/0; agreement: 86%.

For each query, we compared the assigned depth with the best-performing fixed depth, counting ties as agreement. This is a retrospective comparison based on separate executions, not a measure of the true optimal depth.

Most OOLONG queries were assigned Depth-0, helping explain the routing-only score of 57.00%, compared with 65.97% for fixed Depth-2.

Routing-only also cost less on OOLONG: $0.0415 versus $0.0750 per task for Depth-2. Router inference is included in the total costs in Table 2.

We updated Section 6.2 and Figure 2 accordingly.

  1. Verification selection, outcomes, and cost

The implementation prioritizes conflicting observations, uncertain labels, and decisions that could change user membership. Candidates are then ranked by their potential effect on the final pair set. At most 50 decisions are checked per task.

Across 20 OOLONG-Pairs tasks, we made 950 verification calls (47.5/task). Of these:

  • 939 confirmed existing classifications.
  • 5 corrected wrong classifications.
  • 2 changed correct classifications to wrong.
  • 4 responses were unparseable.

Verification added $0.008135 per task on average, counting input and output tokens.

These are observed verifier outcomes, unlike the counterfactual corrections in Q1. Details have been added to Sections 4.2 and 6.3.

  1. Uncertainty and reporting

We computed 95% paired bootstrap confidence intervals using 30,000 resamples of matched task IDs.

Full IR compared with:

  • OOLONG-Pairs Depth-1: +10.55 pp, CI [+3.27, +18.26], 17 wins/3 losses.
  • OOLONG-Pairs Depth-2: +7.77 pp, CI [-0.89, +17.09], 13 wins/7 losses.
  • CodeQA Depth-1: +2.00 pp, CI [-8.00, +12.00], 4 wins/43 ties/3 losses.
  • CodeQA Depth-2: +4.00 pp, CI [-8.00, +16.00], 6 wins/40 ties/4 losses.

These intervals reflect variation across tasks, not independent model runs. The OOLONG-Pairs comparison against Depth-2 includes zero, so the improvement remains uncertain under this analysis. Additional comparisons are in Appendix B.

The LongCoT-mini overall accuracy of 44.90% was mistakenly retained after a scoring correction. The correct value is 46.00% (23/50), matching the domain results. Table 4 has been corrected.

Reviewer_petXOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Informed Recursion which improves Recursive Language Models performance by adapting recursion depth to query structure and selectively verifying intermediate decisions. Authors perform experiments comparing RLM with the proposed methods on various benchmark datasets showing that their method provides higher accuracy numbers for roughly same cost as RLMs.

Critical strengths

N/A

Critical weaknesses

I found the paper hard to read. The introduction looked very poor. The introduction doesn't give any picture on what is going to come in the paper. I have never read such a poor introduction. Looking at the initial sections, either the authors wrote this paper in a hurried fashion or the paper might have been significantly written by AI.

There have been multiple instances where the flow of the paper looked arbitrary. I would encourage the authors to carefully go through the paper and make a major revision to it.

Although the experimental numbers look interesting, the writing is very poor.

Questions for authors

See weakness

1 comment
AuthorsOpenReview ↗

Response to Feedback on Manuscript Presentation

Thank you for the feedback. We have noted the reviews and made changes to the paper so that it effectively communicates the motivation, methodology, and contributions.

We have revised the manuscript to address these concerns.

In particular, we have:

  1. Re-written the Introduction to establish the problem and provided a concrete motivating example.
  2. Explained the proposed approach, summarised the main findings, and outlined the paper's structure.
  3. Improved the progression of Sections 2-4, connecting the RLM execution setting to the failure analysis and subsequently to the proposed mechanisms.
  4. Clarified the methodology and updated Figure 2 to accurately represent the routing and verification procedures. 5)Revised the Results and Analysis to better explain the experimental findings, their implications, and the limitations of the approach. We have also incorporated additional empirical analyses to strengthen the presentation of our contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.