acceptodds
Under review as a conference paper at ICLR 2027

Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning

Abstract

As agents take on longer and more complex problems, controlling the execution becomes a task in its own right. Each step in the run brings new control choices, like which partial work to build on, whether to start fresh, or when to stop. We introduce agentic meta-reasoning, an inference-time harness that makes these choices an explicit and structured reasoning process. Workers carry out the task-level computation, while a controller consolidates what the run has established, explores next options, assesses what each option is worth under the remaining budget, and dispatches the chosen work with context drawn from persistent memory. Between decisions the controller carries only a compact account of the run rather than replaying its full history. Our baselines span production coding agents and research harnesses, together with a Direct Control Agent using the same workers and compute budget allowance. On ProgramBench, which tests long-horizon agentic capability through program reconstruction, meta-reasoning achieves 71.5% with GPT-5.5 against 58.0% for Codex; with Opus 4.8 it achieves 67.2% against 65.5% for Claude Code. On the other benchmarks, spanning abstract reasoning, multi-domain long-horizon reasoning, and proof generation, it gains between 3.6 and 4.2 points over direct control, averaged across three frontier models. It keeps improving over the tested budget ranges where direct control plateaus, though its overhead can hurt at small budgets. Artifact-graph analysis reveals more reuse of earlier work, higher coverage of correct solutions in most settings, and nonuniform gains in final selection. These results indicate that spending computation on structured control becomes more important as agents scale to longer runs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agentic reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MfppOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Meta-reasoning agent, a harness that makes decision-making a explicit part of test-time improving pipeline. The meta-reasoning agent is seperated into four parts for each cycle: assessing the controller state, propose the next batch of choices (i.e. experiments to run), evaluating these candidates and then dispatching workers for execution. Across 4 benchmarks and 3 models, meta-reasoning agents achieve higher score with matched budgets, demonstrating the benefits of this design decision. Furthermore, it exhibits scaling behavior with inference-time budget, and outperforms direct control agents. I find the artifact-graph analyses also quite interesting, for candidate coverage, quality assessment, and final selection.

Critical strengths
  1. The problem motivation is clear and has significant downstream applications. As agentic tasks get more complex and long-horizon, orchestrating these tasks to be effective and efficient is critical, and the method separates decision-making with workers, which has clear motivation and is a relatively clean concept.
  2. The experiments finding is insightful, providing useful information beyond final scores. I especially like the artifact graph which is a clear depiction of how meta-reasoning agents change the orchestrated structure for test-time improvments.
Critical weaknesses
  1. The staged approach for meta-reasoning agents deserve clear justification. Separating gains from this four-phase pipeline help compared to a one-phase alternative would help justify this component.
Questions for authors
  1. Can you provide an ablation on the four-phase pipeline against a single-stage controller which still maintains the same compact state and retrieval interface?
  2. I would like to better understand how this pipeline performances against classic est-time improving methods. Could the authors provide comparisons, especially empirically, against baselines such as AlphaEvolve and Reflexion?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Mfpp

We thank the reviewer for the thoughtful review, for recognizing the problem's significant downstream applications, and for their appreciation of the artifact-graph analyses. We answer the first question with the requested ablation and the second by explaining how our Meta-Reasoning Agent (MRA) differs from fixed test-time methods.

Staged design (weakness and Q1). We ran the suggested ablation: a single-stage controller (SS-MRA) that keeps MRA's compact state, retrieval interface, workers, and instructions, but makes all decisions in a single agentic loop instead of four stages. MRA scores higher in 8 of the 11 settings, ties in 2, and trails by 2.1 points in 1 (General Response; Table 2, Appendix C). So, in most settings, the four-stage architecture contributes beyond the compact state and retrieval interface.

AlphaEvolve and Reflexion (Q2). We agree that comparisons with established test-time methods help place our results, but two differences make a direct comparison not like for like. First, these methods fix the structure of the computation in advance: Reflexion (Shinn et al., 2023) repeats an attempt–reflect–retry loop, Tree of Thoughts (Yao et al., 2023) runs a fixed tree search, Parallel-Distill-Refine (Madaan et al., 2025) runs a fixed number of draft-and-distill rounds, and AlphaEvolve (Novikov et al., 2025) runs an evolutionary loop. MRA instead decides at each step how many parallel branches to explore or whether to keep refining one line of work, based only on the work produced so far. Fixed-structure methods also depend on per-task settings, such as the number of trials, branches, or rounds, so a comparison at one budget would partly measure those choices. Our related-work section (Appendix A) now covers evolutionary search methods, including AlphaEvolve and ShinkaEvolve (Lange et al., 2025), alongside other fixed-structure methods.

Second, AlphaEvolve and Reflexion are built around feedback on every attempt, whereas MRA receives only the task and decides for itself when and how to check its work. AlphaEvolve needs an evaluator that scores the fitness of every candidate, but our benchmarks give the agent no evaluation signal beyond the task itself. Reflexion can use environment feedback, such as test results, which again gives it information MRA does not receive. With feedback the agent generates itself, Reflexion's loop becomes one of the strategies MRA's controller can already choose, such as asking a worker to revise an earlier attempt given a critique. We therefore compare MRA, at the same call budget, with systems that also organize their computation dynamically without an external evaluator. On all three reasoning benchmarks, MRA scores higher than RLM, a method that manages its own context and dynamically decomposes sub-tasks; on ProgramBench, it scores higher than the production coding agents Claude Code and Codex and the research harness mini-SWE-agent.

Thank you for your response and clarifications. I will maintain my positive rating.

AuthorsOpenReview ↗

Response to Reviewer Mfpp

We thank the reviewer for engaging with our response and for their positive rating. If any concerns remain, we would be glad to address them before the discussion period ends.

Reviewer_GZQtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a training-free agentic meta-reasoning framework that separates computation control from task execution. The proposed controller follows four stages—Assess, Propose, Evaluate, and Dispatch—to evaluate existing work, plan subsequent computation, and allocate workers under a limited budget. The framework also uses persistent memory and a compact controller state to manage previous results. Experiments across four benchmarks and three frontier models show improvements over a Direct Control Agent and several existing agent systems. The paper further analyzes budget utilization, artifact dependencies, correct-candidate coverage, and final answer selection.

Critical strengths

1. This is an important and well-motivated research problem. The paper addresses an important question in long-horizon agentic inference: how should an agent decide what computation to perform next? As agents operate over longer horizons, decisions about whether to continue an existing attempt, verify a result, explore an alternative, or stop can substantially affect final performance. Treating these decisions as an explicit meta-reasoning problem is a meaningful direction, and the proposed framework provides a concrete way to study it.

2. Broad empirical evaluation with useful behavioral analysis. The experiments cover four different benchmarks and three frontier models, with improvements in all 12 matched comparisons against Direct Control. The comparisons with RLM, mini-SWE-agent, Codex, and Claude Code further demonstrate the practical potential of the approach. Beyond final performance, the paper analyzes budget utilization, correct-candidate coverage, selection, and artifact dependencies. These analyses provide useful insights into how agents organize computation over longer runs.

Critical weaknesses

1. The necessity of the four-stage controller is not sufficiently demonstrated. The comparison with Direct Control changes not only the four-stage decomposition but also state representation, memory access, and controller instructions, making it difficult to identify the specific benefit of separating Assess, Propose, Evaluate, and Dispatch. A single controller with the same state information, memory access, and decision rules would provide a more convincing comparison. Moreover, all experiments use frontier models for both the controller and workers, without examining whether such strong controllers are necessary. Previous work, including Nielsen et al., Learning to Orchestrate Agents in Natural Language with the Conductor (ICLR 2026), and Xu et al., TRINITY: An Evolved LLM Coordinator (ICLR 2026), has demonstrated that smaller, trained models can handle meaningful agent coordination decisions, although their roles differ from those studied here. An experiment replacing the controller with a smaller 7B–9B model while keeping frontier workers fixed would help clarify the controller's capability requirements and the practical value of the proposed design.

2. Performance improvements may be partly explained by different stopping behavior rather than better computation decisions. Despite having the same nominal model-call budget, Meta-Reasoning often consumes substantially more calls than Direct Control. For example, on ProgramBench with GPT-5.5, Direct Control uses only about 18% of the 1200-call allowance, while Meta-Reasoning uses nearly the entire budget. Appendices F and G also show that Meta-Reasoning follows more conservative stopping criteria, encouraging continued computation when unresolved issues remain. Given the relatively small performance improvements in several model–benchmark settings, it is unclear how much of the gain comes from better computation allocation rather than simply continuing longer. Although Section 4.2 shows that increasing Direct Control's budget does not necessarily improve performance, this does not address the difference in stopping policies. Testing Direct Control with the same stopping criteria, or allowing it to continue after an early stopping decision, would help determine whether the reported gains genuinely arise from more effective meta-reasoning.

Questions for authors

Q1. Could the authors compare the proposed method with a single controller that performs Assess, Propose, Evaluate, and Dispatch together, while retaining the same memory access, state information, and decision rules? This would help establish whether separating these decisions into four stages provides an independent benefit.

Q2. How sensitive is the method to the controller's model capability? For example, could the authors keep the frontier workers fixed and replace the four-stage controller with a smaller model, such as a 7B–9B LLM? This would help clarify whether frontier-level reasoning is necessary for computation control.

Q3. How would the results change if Direct Control used the same stopping criteria as Meta-Reasoning? Alternatively, could the authors allow Direct Control to continue after an early stop when budget remains, and report both final performance and actual model-call usage? This would help distinguish better computation allocation from simply performing more computation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer GZQt (1/2): Single-Stage Controller and Controller Capability

We thank the reviewer for the detailed review and for the concrete experiments they suggest, which let us test our claims more directly. We address the three questions with two new experiments, a single-stage controller and a weaker controller, and with an analysis of how each system uses its budget.

Single-stage controller (weakness 1, Q1). The suggested comparison isolates the effect of the four stages more cleanly than the comparison in the paper. We ran it with a single-stage controller (SS-MRA) that performs Assess, Propose, Evaluate, and Dispatch together while retaining the memory access, state information, and decision rules of our Meta-Reasoning Agent (MRA). MRA scores higher in 8 of the 11 settings, ties in 2, and trails by 2.1 points in 1 (General Response; Table 2, Appendix C). The four-stage architecture therefore provides an independent benefit in most settings.

Controller capability (weakness 1, Q2). This is an interesting question, and we agree that controller capability matters for practical use. Although it is not the focus of our paper, we tested it on two reasoning benchmarks, IMO ProofBench-Advanced and ARC-AGI-2. Instead of a 7–9B model, we used Gemini 3.1 Flash-Lite as the controller and kept Gemini 3.1 Pro as the workers. Because the two models share a family and generation, they differ mainly in capability. We compare this setup with the Direct Control Agent (DCA) and MRA, both of which use Gemini 3.1 Pro as controller and workers:

SystemControllerIMO ProofBench-AdvancedARC-AGI-2
DCAGemini 3.1 Pro82.777.5
MRAGemini 3.1 Flash-Lite76.781.7
MRAGemini 3.1 Pro91.384.2

With the same workers, the weaker controller does not reach the performance of MRA with a Pro controller on either benchmark. On IMO ProofBench-Advanced, it trails MRA by 14.6 points and also falls below DCA. On ARC-AGI-2, it trails MRA by 2.5 points but still outperforms DCA. This is consistent with a central point of our paper: meta-reasoning is more than coordinating workers, since deciding what to do next depends on judging the work they do. A weaker controller may judge less reliably, which matters more on IMO ProofBench-Advanced, where verifying a proof takes strong reasoning, than on ARC-AGI-2, where checking a candidate may be relatively easier. The experiment covers one model family and two benchmarks, so firm conclusions about controller capability would need runs across more benchmarks and model combinations. Conductor and TRINITY, which we already cite in our related work (Appendix A), show that small trained models can coordinate agents, but whether a small model can perform, or be trained to perform, this kind of meta-reasoning effectively remains open. We leave a full evaluation of controller size, and the training of a dedicated meta-reasoning model, to future work.

AuthorsOpenReview ↗

Response to Reviewer GZQt (2/2): Stopping Behavior

Stopping behavior (weakness 2, Q3). We take this concern seriously. MRA does use more of its budget than DCA (Figure 3), and Appendix E acknowledges that, because each system decides how much of its budget to use, our comparisons fix the budget rather than the amount spent. The question is therefore whether MRA's gains come from spending more or from how it spends, and the single-stage ablation lets us test this: SS-MRA uses a prompt nearly identical to MRA's, with the same stopping criteria. On ProgramBench, SS-MRA's workers make 90–113% as many calls as MRA's, yet MRA scores 3.3–4.8 points higher with all three models:

ProgramBenchSS-MRA worker callsMRA worker callsSS-MRA scoreMRA score
Gemini 3.1 Pro92091743.948.7
GPT-5.51,02290367.071.5
Opus 4.89591,06763.967.2

On the reasoning benchmarks, SS-MRA gives its workers fewer calls than MRA in seven of eight settings despite the same stopping criteria; in the eighth, IMO with Opus 4.8, it gives them more and still scores lower:

Reasoning benchmarksSS-MRA worker callsMRA worker callsSS-MRA scoreMRA score
IMO, Gemini 3.1 Pro6.810.486.391.3
IMO, GPT-5.517.023.094.694.6
IMO, Opus 4.833.219.273.280.2
ARC, Gemini 3.1 Pro3.34.577.584.2
ARC, GPT-5.58.321.278.379.2
ARC, Opus 4.86.87.180.080.0
LongCoT-mini, GPT-5.510.023.958.665.1
LongCoT-mini, Opus 4.810.012.668.666.5

Thus, with the same stopping criteria, SS-MRA spends less than MRA on the reasoning benchmarks, and with comparable worker compute on ProgramBench, it still scores lower. We argue that knowing when to stop is itself an important capability, especially as agents run increasingly autonomously, with ever-longer task horizons. To make an agent continue past its own decision to stop, someone outside the agent would have to set how much it should spend, which an autonomous agent should decide for itself. MRA's advantage therefore lies in the quality of decisions its staged controller makes: which work is worth doing, and when to stop.

AuthorsOpenReview ↗

Response to Reviewer GZQt

We thank the reviewer again for their detailed review. We would be grateful to know whether our response addresses their concerns, and we are happy to answer any further questions before the discussion period ends.

Reviewer_QftfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The paper designs a harness to help models on longer-horizon reasoning tasks inspired by the cognitive science idea of meta-reasoning. Rather than keeping all reasoning in a single model context, the meta-reasoning agent breaks down a generation into steps, creates an artifact based on each step, and uses those artifacts as inputs for later reasoning steps. The authors show that this performs better than keeping all of the generations in a single model context in most cases, and outperforms recursive language models at reasoning tasks and codex and claude code on programbench.

Critical strengths
  • Long-horizon reasoning and programming tasks are of interest to the broader community, so studying harness engineering approaches to improve performance is an interesting problem.
  • The agentic meta-reasoning approach presented in this work is successful on the evaluated benchmarks, improving over commercial harnesses.
Critical weaknesses

The major weaknesses of this work are unclear significance and a lack engagement with prior work. The contribution of this works seems to be the idea that agent harnesses can facilitate models checkpointing intermediate work, and launching workers to solve sub-problems. This sounds like a sensible generic agent strategy, but I’m unsure that this contribution is significant enough to warrant recommending acceptance to ICLR. I think this concern could be addressed by the authors contextualizing their work with a dedicated related work section and adding additional baselines. For example, the strategy presented here seems like a reasonable context management strategy, so it would be useful to explicitly discuss how the work relates to and how the results compare to other works such as: Zhou et al., 2025, Light et al., 2026, Li et al., 2026, and Zhang et al., 2025 and in products like cursor: Cassano and Rush, 2026.

Questions for authors
  • In the ProgramBench leaderboard, the task seems to be evaluated on complete program re-creation. Is it standard to use partial test results to compare systems on ProgramBench?
  • Comments: For Figure 2 - it would be clearer to 1) present bar plots with a 0 y-axis, similar to Figure 4. Additionally, including more than one run would provide a sense of variance of results.
4 comments
AuthorsOpenReview ↗

Response to Reviewer Qftf (1/2): Significance and Related Work

We thank the reviewer for the careful review and for the specific pointers to related work, which have improved the paper. Below, we clarify the significance of our contribution, add the suggested works and a new baseline, report ProgramBench under its resolved and almost-resolved metrics, and revise Figure 2.

Significance. The reviewer raises an important question about significance. We agree that storing intermediate work and launching workers on sub-problems is a generic strategy, and our main baseline already uses it: the Direct Control Agent (DCA) has the same workers, artifact memory, and interfaces for choosing which earlier outputs each worker sees (Section 2.4). Our main contribution is the Meta-Reasoning Agent (MRA), which changes how the controller decides what to compute next. Most agents, DCA included, make every decision in one conversation over the accumulated history. MRA instead divides each control cycle among four stages, each a separate agentic loop that starts from a fresh context in every cycle, has its own prompt, and performs one function of meta-reasoning (Section 2.3). With the same workers and budget, MRA scores higher than DCA in all 12 matched comparisons (Table 1).

Following the reviewer's suggestion to add baselines, we have added one that isolates this architecture's contribution: a single-stage variant of MRA (SS-MRA), which keeps MRA's workers, compact state, and memory but makes all decisions in a single agentic loop. Unlike the DCA comparison, it separates the effect of the four stages from that of the compact state and memory. MRA scores higher in 8 of the 11 settings, ties in 2, and trails by 2.1 points in 1 (General Response; Table 2, Appendix C), so in most settings its gains come not only from context management but also from the four-stage architecture we propose.

The paper also contributes methods for diagnosing agent runs beyond final scores. We record each run as an artifact graph, whose nodes are stored outputs and whose edges mark which earlier outputs each one used (Section 3), and show how different measures on this graph explain agent behavior. Coverage and selection (Brown et al., 2024), computed over every candidate in the graph, tell whether a failed run never found a correct answer or found one and submitted another. Type-2 AUC (Fleming and Lau, 2014), applied to judgments during the run, tells whether the workers or the controller better recognize correct work. We also introduce frontier selection gain, which tells whether the final choice beats a random pick among the most developed candidates. These measures show, for example, that MRA finds correct candidates more often in most settings, while its gains in choosing among them vary by model (Section 4.4).

Related work. We agree that the works the reviewer lists are related and have added all of them to our related-work section, which is in Appendix A because of the page limit; we will move the section into the main text for the camera-ready. These works focus on what a model keeps in its context, either by training the model (MEM1, Neural Garbage Collection, Composer) or by evolving the context across tasks (ACE). Context management is not our primary focus: MRA performs meta-reasoning, deciding which computations to run, what each worker sees, and when to stop, and it manages context only in service of these decisions. Our evaluation already includes systems that manage their own context with the same models as MRA (RLM, Claude Code, and Codex), and MRA scores higher than each in every setting where we ran it.

AuthorsOpenReview ↗

Response to Reviewer Qftf (2/2): ProgramBench Metrics and Comments

ProgramBench metric. This is a fair question. ProgramBench (Yang et al., 2026) reports how many programs pass all hidden tests (resolved) and how many pass at least 95% of them (almost resolved); our paper reports the mean test-pass rate. At our budgets no system resolves more than 2 of the 200 programs, so resolution alone cannot separate the systems. Partial results have precedent: the leaderboard publishes per-instance pass rates, and recent work reports the fraction of hidden tests passed (Mao et al., 2026; Claude Fable 5.2 system card). Table 3 (Appendix C), reproduced below, reports all three metrics. MRA ranks first on every metric with GPT-5.5 and Opus 4.8; with Gemini 3.1 Pro, DCA has two more programs at or above 95%.

ModelSystemResolved≥95%Mean test-pass rate
Gemini 3.1 Promini-SWE0242.0
DCA0446.9
MRA0248.7
GPT-5.5mini-SWE0757.6
Codex0458.0
DCA1763.7
MRA22971.5
Opus 4.8mini-SWE01764.7
Claude Code01765.5
DCA02665.3
MRA22967.2

Figure 2 and variance. We have followed the first suggestion and redrawn Figure 2 with the y-axis starting at zero. We agree that repeated runs would show the variance of each result, and Appendix E states this limitation. Repeating every run was beyond our compute budget: we evaluate MRA and DCA at three budget levels on four benchmarks with three models, and a single repetition of the main comparison would allow up to 1.8 million model calls. We therefore base our conclusions not on any single difference but on their consistency: MRA scores higher than DCA in all 12 matched settings, across four benchmarks and three models.

Thank you for the response and clarifications. I have updated my score.

AuthorsOpenReview ↗

Response to Reviewer Qftf

We thank the reviewer for engaging with our response and for updating their score. If any concerns remain, we would be glad to address them before the discussion period ends.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading. We are encouraged that all three reviewers found the problem worth studying (Qftf, GZQt, Mfpp), that meta-reasoning is a meaningful direction (GZQt), and that separating decision-making from execution is well motivated and conceptually clean (Mfpp). Reviewers also found the evaluation broad (GZQt) and the method effective against both research harnesses and commercial agents (Qftf, GZQt). We further appreciate that reviewers found the artifact-graph analyses insightful beyond final scores (GZQt, Mfpp). Reviewers GZQt and Mfpp asked whether the four-stage architecture matters on its own, and Reviewer Qftf asked for additional baselines. We address both with the single-stage ablation below, then summarize the revisions that the reviews led to.

Single-stage ablation. Following the reviewers' suggestions, we added an ablation that isolates the effect of the four stages more cleanly than our comparison with the Direct Control Agent (DCA). The single-stage variant (SS-MRA) of our Meta-Reasoning Agent (MRA) keeps everything in MRA except the separation into stages: the same workers, the same compact state, the same artifact memory and retrieval interface, and the same instructions, which merge the four stage prompts into one. SS-MRA therefore has the same state information, memory access, and decision rules as MRA, and differs only in whether its controller decides in four separate agentic loops or in one.

ModelSystemIMOARCLongCoT-miniProgramBench
Gemini 3.1 ProDCA82.777.553.546.9
SS-MRA86.377.5–43.9
MRA91.384.262.748.7
GPT-5.5DCA93.375.864.763.7
SS-MRA94.678.358.667.0
MRA94.679.265.171.5
Opus 4.8DCA78.077.565.365.3
SS-MRA73.280.068.663.9
MRA80.280.066.567.2

MRA scores higher than SS-MRA in 8 of the 11 settings, ties it in 2, and trails it by 2.1 points in 1 (Opus 4.8 on LongCoT-mini). We could not run Gemini 3.1 Pro on LongCoT-mini: since our submission, our infrastructure for serving the Gemini API has limited each request to 10 minutes, and Gemini's reasoning calls on LongCoT-mini exceed that limit. The results show that, in most settings, the four-stage architecture is an important source of MRA's performance.

Summary of revisions.

  • Appendix C.2 describes the single-stage ablation and reports it in Table 2, Appendix H gives its prompts, and Section 2.4 references it (Qftf, GZQt, Mfpp).
  • Table 3 (Appendix C) reports ProgramBench under resolved, almost resolved, and mean test-pass rate (Qftf).
  • Our related-work section (Appendix A) now covers MEM1, Neural Garbage Collection, Cursor's Composer, Deep Reasoning, and ACE (Qftf), as well as evolutionary search methods such as AlphaEvolve and ShinkaEvolve (Mfpp).
  • Figure 2 now uses a y-axis that starts at zero (Qftf).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.