Long-horizon research requires effective meta-reasoning, i.e., deciding what to investigate next as evidence accumulates. Learning this capability is challenging because the relevant decisions are embedded within long execution traces and their consequences are delayed. We introduce Meta-reasoning for Iterative Research Agents (MIRA), a hierarchical research harness whose outer loop curates context from a persistent Git-based research state and selects the objective for the next inner-loop execution agent. MIRA improves how models allocate additional inference: on IMOProofBench Advanced, it raises GPT-5.5's performance from 67.1% to 100% and helps it explore and pivot to new proof directions; in neural-architecture hill-climbing, it discovers improvements to both Residual Matrix and Loop Transformers. Isolating these decision boundaries also enables targeted credit assignment. We pretrain the language model as a generative critic that forecasts expected remaining return from partial research states across environments, then use the same model as both actor and its own critic in single-rollout asynchronous policy optimization over meta-reasoning decisions. We call this shared model MIRA-AC. By jointly training Qwen3.6-27B-Instruct to value the future potential of a research state and decide what to investigate next, while focusing policy-gradient updates on decisions comprising roughly 25% of generated output, MIRA-AC achieves gains of up to 10–15% across several autoresearch environments. It does so while training on readily available public hill-climbing signals, learning from its own research through proxy feedback rather than sparse or costly final-evaluation labels. Overall, MIRA-AC demonstrates that the strategy for conducting research can itself become a learned, reusable policy.
Thank you for the review, and for noting that MIRA-AC learns from the expected return of a research decision rather than only its immediate outcome, and that the experiments span multiple research settings and models. Please find our answers to your two concerns below.
W1. Lack of comparison under different RL frameworks.
A1. Thank you for raising this. In autoresearch, one rollout is a full research episode: hours of execution on real compute. Methods that need several rollouts per state, such as GRPO-style group sampling, are infeasible at this cost, so being able to learn effectively from single rollouts on a variety of problems is essential. MIRA-AC does this by exposing research decisions as explicit meta-reasoning steps and learning a value for the research state at each one, so the policy learns from the difference between what it expected and what its actions produced. For the same reason, we compare frameworks within single-rollout policy optimization.
The closest existing framework is single-rollout optimization with a token-level value function, as in SAO (Hou et al., 2026), and our Token regression conditions are its analogue. Table 1 then varies how single-rollout optimization is instantiated while focusing learning on meta-reasoning decisions: credit granularity (token- vs. decision-level value), critic parameterization (scalar regression vs. generative), feedback (private-binary vs. public-marginal), and parameter sharing (separate vs. shared). Under the same public-marginal feedback, MIRA-AC improves the primary outcome over Token regression: Symbolic Regression 60 vs. 55 solved, Physics Discovery 10 vs. 6, and HillClimbBench-CPU 58 vs. 49 wins (mean score 0.805 vs. 0.762).
W2. Insufficient evidence of scalability and generalization.
A2. Thank you for raising this point. MIRA operates across many kinds of autoresearch environments: theorem proving, neural-architecture hill climbing, symbolic regression, physics discovery, causal discovery and CPU architecture design. For generalization, our submission evaluates the critic on task-disjoint splits (Sec. 3) and under leave-one-environment-out pretraining, which improves prediction and adaptation on the held-out environment (Fig. 6). This shows the scalability of pretraining value functions across diverse autoresearch environments as a transferable prior for partial research progress: with only five episodes from a new environment, the pretrained critic reaches 0.053 lower RMSE than a critic trained on that environment alone.
For the policy, we evaluate both settings where we aim to train a strong research policy for a particular autoresearch environment, and settings where we aim to train a generally good research policy across many autoresearch environments, which may reward different research behaviors (Table 1). We observe that training meta-reasoning ability introduces desirable qualitative improvements in the model's research process across environments. Across 400 matched trajectory pairs from four environments, a blinded LLM judge favors MIRA-AC's research decisions in 60.8% of decisive judgments (203 vs. 131; 66 tied or unclear), and all six research behaviors it tracks shift in the desirable direction. For example, trajectories that run an experiment discriminating between competing ideas rise from 16.8% to 28.0% (in Physics Discovery, varying one factor to separate candidate laws), while local-search stagnation, i.e., repeatedly trying minor variants of an unproductive idea, falls from 33.5% to 18.8% (in Symbolic Regression, the policy changes the functional form instead of tuning coefficients).
Our eventual goal is a general research policy, and this work develops the components it needs: a transferable prior for valuing partial research progress, pretrained across environments, that improves forecasting and adaptation on held-out ones (Sec. 3); research decisions as explicit units for credit assignment in long-horizon research traces; and a way to focus learning on the meta-reasoning decisions that shape research allocation, making long-horizon reinforcement learning more feasible (Sec. 4).
We hope our responses clarify our choice of single-rollout optimization and our evidence for generalization, and we would appreciate it if you would consider increasing your score. Thank you!