acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents

Abstract

Long-horizon research requires effective meta-reasoning, i.e., deciding what to investigate next as evidence accumulates. Learning this capability is challenging because the relevant decisions are embedded within long execution traces and their consequences are delayed. We introduce Meta-reasoning for Iterative Research Agents (MIRA), a hierarchical research harness whose outer loop curates context from a persistent Git-based research state and selects the objective for the next inner-loop execution agent. MIRA improves how models allocate additional inference: on IMOProofBench Advanced, it raises GPT-5.5's performance from 67.1% to 100% and helps it explore and pivot to new proof directions; in neural-architecture hill-climbing, it discovers improvements to both Residual Matrix and Loop Transformers. Isolating these decision boundaries also enables targeted credit assignment. We pretrain the language model as a generative critic that forecasts expected remaining return from partial research states across environments, then use the same model as both actor and its own critic in single-rollout asynchronous policy optimization over meta-reasoning decisions. We call this shared model MIRA-AC. By jointly training Qwen3.6-27B-Instruct to value the future potential of a research state and decide what to investigate next, while focusing policy-gradient updates on decisions comprising roughly 25% of generated output, MIRA-AC achieves gains of up to 10–15% across several autoresearch environments. It does so while training on readily available public hill-climbing signals, learning from its own research through proxy feedback rather than sparse or costly final-evaluation labels. Overall, MIRA-AC demonstrates that the strategy for conducting research can itself become a learned, reusable policy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Autoresearch, meta-reasoning, recursive self-improvement, LLM, research agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cNqpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposed MIRA as a hierarchical architecture to separate research decision from execution. Then it extends the framework with reinforcement learning, to optimize the research objective decision by estimating the expected progress based on next token distribution.

Critical strengths
  1. The paper applies reinforcement learning framework into the research decision making, which is not only focused on current outcome, but predict the expected return if the decision is taken.
  2. The paper conduct experiments across multiple research settings with different llm models.
Critical weaknesses
  1. Lack of comparison under different RL frameworks.
  2. Insufficient evidence of scalability and generalization.
Questions for authors

N/A

1 comment

Thank you for the review, and for noting that MIRA-AC learns from the expected return of a research decision rather than only its immediate outcome, and that the experiments span multiple research settings and models. Please find our answers to your two concerns below.

W1. Lack of comparison under different RL frameworks.

A1. Thank you for raising this. In autoresearch, one rollout is a full research episode: hours of execution on real compute. Methods that need several rollouts per state, such as GRPO-style group sampling, are infeasible at this cost, so being able to learn effectively from single rollouts on a variety of problems is essential. MIRA-AC does this by exposing research decisions as explicit meta-reasoning steps and learning a value for the research state at each one, so the policy learns from the difference between what it expected and what its actions produced. For the same reason, we compare frameworks within single-rollout policy optimization.

The closest existing framework is single-rollout optimization with a token-level value function, as in SAO (Hou et al., 2026), and our Token regression conditions are its analogue. Table 1 then varies how single-rollout optimization is instantiated while focusing learning on meta-reasoning decisions: credit granularity (token- vs. decision-level value), critic parameterization (scalar regression vs. generative), feedback (private-binary vs. public-marginal), and parameter sharing (separate vs. shared). Under the same public-marginal feedback, MIRA-AC improves the primary outcome over Token regression: Symbolic Regression 60 vs. 55 solved, Physics Discovery 10 vs. 6, and HillClimbBench-CPU 58 vs. 49 wins (mean score 0.805 vs. 0.762).

W2. Insufficient evidence of scalability and generalization.

A2. Thank you for raising this point. MIRA operates across many kinds of autoresearch environments: theorem proving, neural-architecture hill climbing, symbolic regression, physics discovery, causal discovery and CPU architecture design. For generalization, our submission evaluates the critic on task-disjoint splits (Sec. 3) and under leave-one-environment-out pretraining, which improves prediction and adaptation on the held-out environment (Fig. 6). This shows the scalability of pretraining value functions across diverse autoresearch environments as a transferable prior for partial research progress: with only five episodes from a new environment, the pretrained critic reaches 0.053 lower RMSE than a critic trained on that environment alone.

For the policy, we evaluate both settings where we aim to train a strong research policy for a particular autoresearch environment, and settings where we aim to train a generally good research policy across many autoresearch environments, which may reward different research behaviors (Table 1). We observe that training meta-reasoning ability introduces desirable qualitative improvements in the model's research process across environments. Across 400 matched trajectory pairs from four environments, a blinded LLM judge favors MIRA-AC's research decisions in 60.8% of decisive judgments (203 vs. 131; 66 tied or unclear), and all six research behaviors it tracks shift in the desirable direction. For example, trajectories that run an experiment discriminating between competing ideas rise from 16.8% to 28.0% (in Physics Discovery, varying one factor to separate candidate laws), while local-search stagnation, i.e., repeatedly trying minor variants of an unproductive idea, falls from 33.5% to 18.8% (in Symbolic Regression, the policy changes the functional form instead of tuning coefficients).

Our eventual goal is a general research policy, and this work develops the components it needs: a transferable prior for valuing partial research progress, pretrained across environments, that improves forecasting and adaptation on held-out ones (Sec. 3); research decisions as explicit units for credit assignment in long-horizon research traces; and a way to focus learning on the meta-reasoning decisions that shape research allocation, making long-horizon reinforcement learning more feasible (Sec. 4).

We hope our responses clarify our choice of single-rollout optimization and our evidence for generalization, and we would appreciate it if you would consider increasing your score. Thank you!

Reviewer_a5z9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how long-horizon research agents can learn to decide what to investigate next. It introduces MIRA, a hierarchical framework in which an outer model curates evidence from a persistent Git repository and issues work orders to fresh execution agents. MIRA-AC extends this framework with a shared generative actor–critic that predicts remaining research progress and optimizes work-order and stopping decisions, including training from public progress signals. Training reconstructs complete research episodes and uses asynchronous actor and critic updates. Validation covers IMOProofBench Advanced, RMT and Loop Transformer architecture search, task-disjoint and cross-environment critic prediction, and policy learning in symbolic regression, physics discovery, Bayesian causal discovery, and CPU architecture tasks. The paper reports inference-time improvements, improved value prediction, and policy gains under private evaluation, with variation across environments and training configurations.

Critical strengths
  1. A useful decision interface backed by substantive inference-time results. MIRA exposes research allocation as explicit work orders over persistent repository state, while treating execution as an environment transition. This gives a concrete boundary for context management and subsequent credit assignment (Sections 2 and 4). The evaluation includes a particularly valuable token-matched control: on IMOProofBench Advanced, forced continuation brings GPT-5.5 and Codex to mean normalized grades of 93.3% and 90.0%, respectively, while MIRA reaches 100% across all 30 problems in each of three trials (Figure 2(a)). Thus, most of the natural-stopping gap is explained by additional inference, but a meaningful residual remains. The RMT and Loop campaigns also produce architectures that improve private evaluation, beyond the public objectives used during search (Sections 2.3, Appendices F–G). These results establish practical value across proof construction and executable research, although the architecture campaigns are individual case studies.

  2. The decision-level learning design has supporting evidence beyond final policy performance. MIRA-AC trains the outer model to generate work orders and predict remaining research progress, applying policy loss to work-order and stopping decisions while leaving executor generations outside that loss. Section 3 evaluates the critic under task-disjoint support/query splits: the decision-level generative critic obtains RMSE 0.214, compared with 0.324 for the token-level generative alternative and 0.222 for a decision-level categorical head. The evidence is therefore stronger for decision-level value estimation than for a large additional benefit from generative parameterization. Leave-one-environment-out pretraining also improves prediction and adaptation, including a reported 0.053 RMSE advantage after five target support episodes (Figure 6). These experiments provide evidence of generalization for value prediction separately from the more limited policy evaluation. The implementation specifies complete-episode reconstruction and critic-version handling across asynchronous boundaries (Appendices I–J). Table 1 adds encouraging downstream results: public-marginal MIRA-AC improves symbolic-regression successes from 50 to 60 and HillClimbBench-CPU mean private score from 0.7624 to 0.8054. Together, these results support the proposed research-decision boundary as a useful unit of learning.

Critical weaknesses
  1. The reliability and scope of learned strategy remain incompletely established. Appendix I explicitly reports one trained policy per configuration. Appendix C further states that training and evaluation in the three scientific environments use the same task identities and seeds, with independently initialized research trajectories. This supports improvement on new trajectories of familiar tasks; it gives limited evidence about reusable strategy on new problems. The instability is consequential: public-marginal single-environment MIRA-AC reduces BNLearn successes from 76 to 48, and private-binary MIRA-AC also reduces Physics Discovery successes from 8 to 6 (Table 1). The mixed-environment policy improves all four environments, including BNLearn from 76 to 84, which is encouraging; however, BNLearn participates in critic pretraining, and several gains are modest in absolute counts. The authors appropriately describe this as actor-learning transfer with a previously exposed value domain. Independent training repetitions and uncertainty for the paired policy comparisons would make it much easier to assess how dependable these gains are. Task-level bootstrapping cannot establish variability across training runs.

  2. The experiments establish the combined MIRA-AC configuration more clearly than the proposed explanation for its advantage. Separate actors start from the base policy, whereas MIRA-AC starts from a critic-pretrained checkpoint; Appendix I also lists different LoRA target modules for these configurations. Section 4.3 explicitly acknowledges the initialization difference, so this is a limitation of attribution rather than an undisclosed experimental flaw. Nevertheless, the stronger interpretation in Section 4.4—that the shared critic loss improves the policy representation and thereby explains the advantage—remains underdetermined. The pretrained actor initialization and the adapter parameterization can themselves affect policy quality. An evaluation of the critic-pretrained checkpoint acting before online updates, together with an initialization- and module-matched separate actor–critic, would distinguish the benefit of online co-training from these alternatives. This matters because learning through the shared value objective is a central methodological claim.

Questions for authors
  1. Clarification affecting the cross-environment comparison: For the final row of Table 1, what are the total training episodes, their allocation across the three source environments, and the resulting actor/critic update counts? Do the stated 100 episodes apply to this run in total or separately per environment? This determines whether its improvements can be interpreted primarily as a benefit of mixed-environment training or also involve additional online training.

  2. Validation of the current co-training claim: How does the critic-pretrained checkpoint perform as an actor before online updates? With actor initialization and LoRA target modules matched, does shared MIRA-AC still improve over the separate generative actor–critic? These controls would directly affect my assessment of the representation-learning explanation in Section 4.4.

  3. Validation of the current reliability claim: Can the authors provide independent policy-training repetitions for the public-marginal shared and mixed-environment configurations, together with paired task-level uncertainty? The key issue is whether the mixed policy's gains and the single-environment BNLearn collapse recur, since these outcomes currently drive opposite conclusions about the reliability of learned research allocation.

3 comments

Thank you for the constructive review, and for highlighting the decision interface (which treats execution as an environment transition and gives a concrete boundary for credit assignment), the token-matched control, and the evidence for decision-level learning beyond final policy performance. We ran the controls you proposed; please find our answers to your three questions below.

Q1. Clarification affecting the cross-environment comparison: For the final row of Table 1, what are the total training episodes, their allocation across the three source environments, and the resulting actor/critic update counts? Do the stated 100 episodes apply to this run in total or separately per environment? This determines whether its improvements can be interpreted primarily as a benefit of mixed-environment training or also involve additional online training.

A1. Thank you for asking for this clarification. The stated 100 episodes apply separately per environment. The cross-environment run makes one pass over 300 episodes in total (100 from each of Symbolic Regression, Physics Discovery and HillClimbBench-CPU), and performs 49 actor updates and 90 critic updates. For reference, each single-environment run performs 12–23 actor updates and 23–45 critic updates. We have added these counts to App. I.

The cross-environment run therefore receives more online training in total, but it is spread across three very different research regimes, with the same amount per environment as the single-environment runs. Its row is meant to show how well one policy learns from a mixture of regimes, while the single-environment rows show how well the recipe specializes to one.

Q2. Validation of the current co-training claim: How does the critic-pretrained checkpoint perform as an actor before online updates? With actor initialization and LoRA target modules matched, does shared MIRA-AC still improve over the separate generative actor–critic? These controls would directly affect my assessment of the representation-learning explanation in Section 4.4.

A2. Thank you for proposing these controls; we ran both. In short, critic pretraining alone, when its checkpoint is used to initialize the actor, can improve the policy before further online actor updates. With actor initialization and LoRA targets matched, shared MIRA-AC improves over the separate generative actor–critic in single-environment training on HillClimbBench-CPU and BNLearn, and the two perform comparably in cross-environment training. We detail both controls below.

The critic-pretrained checkpoint as an actor. We evaluated the critic-pretrained checkpoint directly as the actor, before any online update, on all four environments:

Symbolic (solved)Physics (solved)HCB-CPU (wins vs. untrained MIRA)BNLearn (solved)
Untrained MIRA508(reference)76
Critic-pretrained, no actor updates5685091
Cross-env MIRA-AC63105584

Trained only as a critic, the shared parameters already improve the policy on BNLearn (91 vs. 76 solved). Online actor training then adds further gains in the environments it trains on: Symbolic Regression improves from 56 to 63, Physics Discovery from 8 to 10, and HillClimbBench-CPU from 50 to 55 wins.

BNLearn receives no actor updates in the cross-environment run, but training on the other three environments still changes how the policy writes work orders there: its follow-up work orders become longer and more prescriptive, so more executor episodes run to the time limit (61% vs. 52%) and progress per iteration is slower. The policy keeps part of its critic-pretraining gain (84 vs. 91 for the critic-pretrained checkpoint); we will study this behavior further and update the paper accordingly.

Initialization- and module-matched separate actor–critic. For this control, we trained separate actor–critics whose actor starts from the same critic-pretrained adapter as MIRA-AC and adapts the same 12 LoRA modules, on the same environments and episodes. The only remaining difference is that actor and critic keep separate parameters.

In single-environment training, shared MIRA-AC improves over the matched separate actor–critic: on HillClimbBench-CPU it reaches 58 vs. 57 wins (mean score 0.805 vs. 0.785), and on BNLearn it solves 83 vs. 77 tasks. In cross-environment training, the two perform comparably (MIRA-AC vs. matched separate): Symbolic Regression 63 vs. 65, Physics Discovery 10 vs. 8, HillClimbBench-CPU 55 vs. 57 wins, and BNLearn 84 vs. 86. Training the same model on both functions can therefore be an effective way to match or improve performance while removing the need to keep a separate critic model in memory. We intend to perform further experiments to study how the gradients of the actor and critic updates interact, in hopes of providing more detailed guidance on co-training the actor and critic objectives.

To locate where the gain over the paper's separate actor–critic comes from, we also changed one factor at a time on HillClimbBench-CPU. All rows are final actors after the same online training, evaluated on the same tasks:

HillClimbBench-CPUWins vs. untrained MIRAMean score
Separate AC, paper (actor from base, 7 modules)530.769
+ actor adapts the critic's 12 modules530.752
+ actor initialized from the critic-pretrained adapter (actor and critic still separate)570.785
+ actor and critic share one adapter throughout training (MIRA-AC)580.805

On HillClimbBench-CPU, matching the LoRA modules did not appear to change the outcome much. Initializing the actor with parameters trained on the value objective provides strong gains in some environments, while online actor training from there is more beneficial in others; together, they yield consistent gains across environments. Our online runs make a single pass over the training episodes, which leaves room for further gains from longer online training.

(Part 3 of 3)

Q3. Validation of the current reliability claim: Can the authors provide independent policy-training repetitions for the public-marginal shared and mixed-environment configurations, together with paired task-level uncertainty? The key issue is whether the mixed policy's gains and the single-environment BNLearn collapse recur, since these outcomes currently drive opposite conclusions about the reliability of learned research allocation.

A3. Thank you for the opportunity to address this. We can confirm that the mixed policy's gains recur across repeated evaluations, and the BNLearn collapse turned out to be an evaluation artifact, which we have corrected. We detail both below.

Mixed-environment gains. We evaluated untrained MIRA and the cross-environment MIRA-AC policy two more times on all four environments, using the same tasks and setup. Research decisions are sampled at temperature 1.0, so each repeated evaluation of the same policy follows a different research trajectory from the first decision onward. We report the mean and standard deviation over the three runs, and the paired gain: the per-task difference between MIRA-AC and untrained MIRA, averaged over the runs, with a 95% bootstrap interval over tasks.

EnvironmentUntrained MIRAMIRA-ACPaired gain [95% CI]
Symbolic (solved)53.3 ± 2.962.3 ± 1.2+9.0 [+3.3, +14.7]
Physics (solved)7.0 ± 1.010.3 ± 0.6+3.3 [+0.3, +6.3]
BNLearn (solved)69.0 ± 6.284.3 ± 0.6+15.3 [+9.0, +21.7]
HCB-CPU (gold score)0.769 ± 0.0090.800 ± 0.013+0.031 [+0.007, +0.058]

As shown in the table above, MIRA-AC improves over untrained MIRA in every environment, and the paired intervals exclude zero in all four. These runs capture evaluation variability for a fixed trained policy. Independent training repetitions are more expensive, since each one is a full online run over 100–300 research episodes; we will include them in the final revision. In the meantime, we have completed an independent trained repetition on a different seed for the public-marginal shared configuration on Symbolic Regression, which solved 63 tasks.

The BNLearn collapse. We audited this evaluation and found that the low result was an evaluation artifact. The two single-environment BNLearn evaluations (public-marginal and private-binary) ran concurrently on shared base-model servers, and the slower serving caused many episodes to time out. With the corrected evaluation, single-environment MIRA-AC solves 83 BNLearn tasks under public-marginal training (previously 48) and 90 under private-binary training (previously 56). We will update Table 1 accordingly, and in the final version we will report all BNLearn configurations with repeated evaluations under identical serving.

We hope our responses have addressed your questions, and we would appreciate it if you would please consider increasing your score. Thank you so much!

Reviewer_DaizOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MIRA, a hierarchical research agent that separates high-level research decisions from execution using persistent Git memory, scoped work orders, and fresh-context executors. It further proposes a decision-level generative critic and MIRA-AC, which jointly trains a shared actor–critic with stitched trajectories and asynchronous policy updates. Evaluation spans theorem proving, architecture search, symbolic regression, physics discovery, and causal discovery. MIRA solves all 30 IMOProofBench Advanced problems in three GPT-5.5 trials, while the cross-environment MIRA-AC policy improves private results over untrained MIRA in all four autoresearch environments (Secs. 2–4; Table 1).

Critical strengths
  1. Clear decision-level formulation: MIRA isolates high-level research choices for targeted credit assignment without optimizing long executor traces. The step-level critic lowers RMSE from 0.324 to 0.214 (Secs. 2–4; Fig. 6a).
  2. Broad empirical validation: The paper evaluates multiple research settings, and cross-environment MIRA-AC improves private outcomes over untrained MIRA across all four autoresearch environments (Table 1).
Critical weaknesses
  1. Limited component attribution: Token-matched continuation recovers much of the theorem-proving gap, while actor–critic comparisons differ in initialization and LoRA targets (Sec. 2.2; App. I).
  2. Limited robustness and generalization evidence: Single-environment MIRA-AC regresses on BNLearn (48 vs. 76 solved under public-marginal training); only one policy is trained per setting, and the evaluations reuse task identities (Table 1; Apps. C, I).
Questions for authors
  1. Can matched ablations isolate the effects of work-order planning, context resets, and actor–critic parameter sharing?
  2. How stable are the results across training seeds and on task-disjoint evaluation splits, especially for BNLearn?
  3. Does BNLearn transfer persist without BNLearn critic pretraining, and how realistic is per-iteration feedback based on the hidden graph’s SHD?
2 comments

Thank you for the careful review, and for recognizing how MIRA isolates high-level research decisions for targeted credit assignment, as well as the decision-level critic and the breadth of our evaluation. Please find our answers to your concerns and questions below.

W1. Limited component attribution: Token-matched continuation recovers much of the theorem-proving gap, while actor–critic comparisons differ in initialization and LoRA targets (Sec. 2.2; App. I).

A1. Thank you for raising these two points. We address the token-matched control first, and then the differences in initialization and LoRA targets.

Token-matched continuation. Token-matched continuation closes most of the aggregate gap, but not the gap on the hardest problems. Six problems were not solved by either GPT-5.5 baseline in any natural-stopping trial. Even with matched tokens, two of these six remain unsolved by both baselines, while MIRA solves all six in each of its three trials. On these problems, MIRA more often explores alternative proof directions instead of repeatedly refining approaches that are not working (Sec. 2.2).

Initialization and LoRA targets. To separate these differences, we ran three controls. First, we evaluated the critic-pretrained checkpoint as an actor before any online update. It already improves the policy on BNLearn (91 vs. 76 solved), and cross-environment online actor training improves it further in the environments it trains on. Second, on HillClimbBench-CPU, we matched the separate actor's LoRA targets to the critic's; the outcome did not appear to change much (53 vs. 53 wins, where a win is a task on which the final result beats untrained MIRA's). Third, we separated the two parts of parameter sharing. In MIRA-AC, actor and critic are a single adapter, so the actor both starts from the critic-pretrained parameters and keeps being updated by the critic loss during online training. Initializing the actor with parameters trained on the value objective provides strong gains in some environments, while online actor training from there is more beneficial in others; together, they yield consistent gains. The full tables are in our response to Reviewer a5z9 (A2), and we will include them in the paper.

W2. Limited robustness and generalization evidence: Single-environment MIRA-AC regresses on BNLearn (48 vs. 76 solved under public-marginal training); only one policy is trained per setting, and the evaluations reuse task identities (Table 1; Apps. C, I).

A2. Thank you for raising these points. We address the BNLearn regression and stability across runs here, and task identities in A4.

BNLearn regression. We audited this evaluation and found that the low result was an evaluation artifact. The two single-environment BNLearn evaluations ran concurrently on shared base-model servers, and the slower serving caused many episodes to time out. With the corrected evaluation, single-environment MIRA-AC solves 83 BNLearn tasks under public-marginal training (previously 48) and 90 under private-binary training (previously 56). We will update Table 1 accordingly, and in the final version we will report all BNLearn configurations with repeated evaluations under identical serving.

Stability across runs. To measure how stable the gains are, we evaluated untrained MIRA and the cross-environment MIRA-AC policy two more times on all four environments, using the same tasks and setup. Research decisions are sampled at temperature 1.0, so each repeated evaluation follows a different research trajectory from the first decision onward. We report the mean and standard deviation over the three runs, and the paired gain: the per-task difference between MIRA-AC and untrained MIRA, averaged over the runs, with a 95% bootstrap interval over tasks.

EnvironmentUntrained MIRAMIRA-ACPaired gain [95% CI]
Symbolic (solved)53.3 ± 2.962.3 ± 1.2+9.0 [+3.3, +14.7]
Physics (solved)7.0 ± 1.010.3 ± 0.6+3.3 [+0.3, +6.3]
BNLearn (solved)69.0 ± 6.284.3 ± 0.6+15.3 [+9.0, +21.7]
HCB-CPU (gold score)0.769 ± 0.0090.800 ± 0.013+0.031 [+0.007, +0.058]

As shown in the table above, MIRA-AC improves over untrained MIRA in every environment, and the paired intervals exclude zero in all four. These runs capture evaluation variability for a fixed trained policy. Independent training repetitions are more expensive, since each one is a full online run over 100–300 research episodes; we will include them in the paper.

Q1. Can matched ablations isolate the effects of work-order planning, context resets, and actor–critic parameter sharing?

A3. Thank you for this question. To isolate work-order planning and context resets, we ran two new ablations, each removing one component from MIRA. We used the paper's Qwen3.6-27B MIRA setup on IMO-ProofBench Advanced (30 problems), with the same 100-iteration cap in every arm.

  • Without work-order planning, we remove the outer model. In each iteration, Codex receives the same fixed instruction (the problem and its iteration number), starts from the repository and its handovers, and decides for itself when to submit.
  • Without executor context resets, the outer model is unchanged, but Codex continues one session across all work orders instead of starting each work order with a fresh context.
IMO-ProofBench AdvancedNormalized score
MIRA43.0%
Without work-order planning37.5%
Without executor context resets37.0%

Scores are means over three grading passes. Removing either component costs MIRA 5.5–6.0 points, so both work-order planning and fresh executor contexts contribute to its performance. Parameter sharing is addressed by the controls in A1. We will include these ablations in the paper.

Q2. How stable are the results across training seeds and on task-disjoint evaluation splits, especially for BNLearn?

A4. Thank you for this question. On stability, the repeated evaluations in A2 show that MIRA-AC's gains hold in Symbolic Regression, Physics Discovery and HillClimbBench-CPU, and its BNLearn result is stable across evaluations (84.3 ± 0.6 solved). We will include independent training repetitions in the paper.

On task identities, our focus in autoresearch is to show that training on the public hill-climbing signal improves the private evaluation of the same research problem. A research problem, such as an architecture to improve or a system to model, is typically worked on repeatedly, so improving on new research trajectories of the same problem is an important objective (App. C). For transfer beyond training tasks, our submission evaluates the critic on task-disjoint splits and held-out environments (Sec. 3), and more thorough exploration of the extent of actor transfer is something we plan to study further.

Q3. Does BNLearn transfer persist without BNLearn critic pretraining, and how realistic is per-iteration feedback based on the hidden graph's SHD?

A5. Thank you for these two questions. We will run training to test actor transfer without critic pretraining on that environment and see how behaviors translate across autoresearch environments in the revision. Per-iteration SHD feedback. In BNLearn, the per-iteration feedback is the directed SHD against the hidden graph, reported once after each iteration (App. C). This setting tests whether the policy can learn from limited, partial-progress access to the objective, while the final score still requires exact recovery of the graph. Motivated by your question, we are also studying how alternative forms of this feedback affect learning, and will include updated results in the revision.

We hope our responses have addressed your concerns, and we would appreciate it if you would please consider increasing your score. Thanks so much!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.