acceptodds
Under review as a conference paper at ICLR 2027

Language Models as AI Research World Models

Abstract

AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet their ability to propose experiments outpaces their capacity to execute them in real environments, making outcome prediction a key capability for sustained self-improvement under limited experimental budgets. We investigate language models as Research World Models (RWMs), which predict the outcomes of candidate interventions across research environments. Our evaluation draws on over 2,600 experimental records from nine research environments spanning pretraining, post-training, and inference, representing more than 171,000 H100 GPU-hours of experimentation. Research knowledge acquired from real experimental experience improves RWM predictions of unseen interventions and can be reused across environments. Across five autoregressive pretraining environments, adding source-environment records without target-environment records reduces macro-average best-of-3 regret (Regret@3) by 52.2% when selecting three of sixteen candidates. These benefits extend to multi-round Autoresearch under fixed selection budgets. Ablations across 13 language models used as RWMs show that adding research knowledge can improve intervention ranking more than changing models or increasing reasoning effort alone. These findings support language models as RWMs and motivate accumulating experimental data for future RWM training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Research World Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jiT5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes using off-the-shelf LLMs as Research World Models which can be used to predict the outcome of a proposed change before evaluating it. They find that providing records of past experiments, from the same or even different environments, substantially improve both LLM forecasts accuracy and candidate selection, and helps much more than using a stronger model or increasing reasoning effort.

Critical strengths
  • The result that experimental records enhance performance much more than model choice or reasoning effort seems to hold across all 13 models.
  • The experimental data is extensive with 2,653 experimental records from nine research environments spanning pretraining, post-training, and inference.
Critical weaknesses

The source of the gains is unclear. The current baselines (TF-IDF kNN and ridge) are weak and cannot establish whether the gains come from LLM reasoning or merely access to the experimental records. Stronger baselines such as embedding-based retrieval or a small model fine-tuned on the same records would help. Ablations on the number of records could also clarify how performance scales with available data.

Questions for authors
  • How were the history and test splits constructed for the in-environment setting? Could closely related interventions appear in both, and if so, how can you distinguish whether the gains reflect generalization to new interventions or partly lookup of closely related ones in the context?
  • How does the in-environment performance change as the number of experimental records varies, and did you test whether giving the LLM only the few most similar records recovers most of the gain?
4 comments

W1: Stronger baselines: embedding-based retrieval

We thank the reviewer for this suggestion. We added embedding-based retrieval baseline and evaluated two ways of using the retrieved records: directly averaging their gains (Embedding-top16 gain average, non-LLM baseline), or providing them to the same LLM for prediction (embedding-top16+LLM).

“Embedding-top16 gain average” achieves 0.62 Spearman. Together with the results of kNN (0.78) and Ridge(0.67) baseline, it indicates that merely having “access to the experimental records” is insufficient to reproduce the LLM’s predictive performance (full320+LLM:0.9). In contrast, “embedding-top16+LLM” achieves 0.87, indicating that the gains come from LLM reasoning.

MethodMAE ↓\downarrowSpearman ↑\uparrow
Embedding-top1 gain3.30e-20.59
Embedding-top16 gain average3.36e-20.62
embedding-top16+LLM2.12e-20.87
full320+LLM2.03e-20.90

W1 & Q2: ablation of experimental record quantity.

We vary the number of historical records in the OLMo3-100M in-env setting, keeping the test set fixed and using Claude Opus 5 (HIGH).

Number of historical records064128192256320
Spearman ↑\uparrow0.620.740.880.890.870.90
MAE (BPB) ↓\downarrow2.93e−22.54e-22.16e-22.08e-22.12e-22.03e-2

In this setting, 128 records already capture most of the predictive improvement (0.88 vs 0.9). Further increases in history size yield smaller, non-monotonic improvements, although MAE continues to decrease overall.

Q1: In-env history/test splits and potential gains from looking up related interventions

We explicitly controlled overlap between highly related interventions when constructing the history and test splits. Within each intervention direction, we connected candidates satisfying both a structured TF-IDF similarity ≥0.90 and a typed intervention distance ≤0.10, and assigned each connected component entirely to the same split. History and test sets contain no cross-split pairs of highly related interventions as defined by these criteria. Regarding whether simple lookup can explain the predictive gains, please see W1 ‘stronger baselines’ table. Comparing the rows of ‘Embedding-top1 gain’, ‘Embedding-top16 gain average’ and ‘embedding-top16+LLM’ shows that most gain comes from generalization rather than direct looking up the table.

Q2: 'LLM+most similar records' vs 'LLM+full records'

See ‘W1: Stronger baselines: embedding-based retrieval’ table. Comparing ‘embedding-top16+LLM’ and ‘full320+LLM’ shows that giving the LLM only the few most similar records recovers most of the gain.

Dear Reviewer, thank you again for your review. We have posted responses with new analyses addressing your concerns Stronger baselines, ablation of experimental record quantity, and In-env history/test splits . We would be grateful to know whether these address your concerns, and we are happy to clarify anything further before the discussion period ends.

Thank you for the answer and new experiments. The embedding-retrieval comparison addresses my main concern on whether the LLM adds substantial value beyond access to the records. The history-size ablation and the description of the split construction are also helpful. Given the new evidence, I am raising my score.

Thank you for your positive assessment and recognition of our work! We are glad that your concern has been resolved. We also sincerely appreciate the time and effort you devoted to reviewing our work and engaging in the discussion.

Reviewer_HASfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the use of LLMs as Research World Models to predict the performance gains of proposed AI research interventions before execution. Using 2,653 real experimental records across nine research environments spanning pretraining, post-training, and inference, the authors demonstrate that historical experimental knowledge improves prediction accuracy and experiment selection. They show that knowledge can transfer across research environments, reducing Regret@3 by 52.2% in cross-environment experiment selection. The results also suggest that providing experimental knowledge can be more beneficial than increasing model capacity or reasoning effort.

Critical strengths
  1. This paper provides valuable empirical evidence for the feasibility of predicting experimental outcomes using LLMs, as the experiments are very solid with 2653 real experimental records representing over 171000 H100 GPU-hours.

  2. The paper shows that historical experiments from other environments can improve predictions and experiment selection even without target-environment records. This is a promising finding for developing general-purpose AI research agents that can accumulate and reuse experimental knowledge.

Critical weaknesses
  1. The cross-environment experiments are restricted to five autoregressive pretraining environments, which demonstrate knowledge transfer between closely related research tasks rather than general transferability across fundamentally different research domains. Would knowledge acquired from pretraining experiments still improve predictions for Code RL or inference optimization?

  2. The baselines kNN and Ridge Regression may be insufficient to establish the advantages of the proposed approach. Stronger LLM-based baselines would be valuable. For example, retrieve relevant historical experiments using embedding similarity and provide them to the same LLM for prediction, or first use an LLM to summarize generalizable insights from historical experiments before predicting new outcomes.

  3. Many evaluated research environments, such as OLMo, are publicly available models and implementations that Claude Opus 5 may have encountered during pretraining. Such inner knowledge may still interact with the provided experimental records, making it unclear how well the observed cross-environment transfer would generalize to genuinely novel research settings. Additionally, I would appreciate clarification on whether Claude Code or other agentic tools were involved in the experimental workflow.

Questions for authors

See Weaknesses

3 comments

W1: Transfer beyond AutoRegressive Pretraining

We thank the reviewer for raising this. Before presenting additional results, we would like to emphasize that autoregressive pretraining already spans a broad range of model architectures and training recipes, and transferring outcome predictions across these settings is itself a challenging problem. To address the reviewer’s question, we further examine 4 target envs: Code RL, inference optimization, math distillation, and diffusion. We compare target-env history alone with the same history augmented by 305 AR pretraining records. This comparison tests whether AR pretraining experience provides additional predictive value.

The table below reports Spearman correlations.

Target env (test set)Target history onlyTarget history + AR historyΔρ\Delta \rho
Code RL (115)0.560.54−0.02
Inference optimization (95)0.890.91+0.02
Math distillation (100)0.750.76+0.01
Diffusion (114)0.770.770.00

We do not observe a consistent incremental benefit from adding AR history across these tasks. One possible explanation is that these tasks differ substantially in their training objectives, intervention spaces, and evaluation metrics, spanning pretraining, post-training, and inference, which may limit the experimental experience that can be directly reused (aligned with human experience).

We will state the boundary precisely in the revision: research knowledge transfers across architectures, recipes, and scale (see response to Reviewer EH6V) within a research family, but not consistently across training paradigms.

W2: Stronger LLM-based baselines

We thank the reviewer for this suggestion. Because our study asks where the predictive gain comes from, we added the suggested variants as controls that decompose the gain:

  • random16: 16 randomly selected historical records;
  • embedding-top16: 16 historical records retrieved by embedding similarity (embedding model: all-MiniLM-L6-v2);
  • history-summary: a summary of the history generated by another LLM.

The kNN and ridge baselines in the paper serve a related role: they test whether the same records, used without an LLM, can reproduce the result.

The experiments cover OLMo3 in-env and Qwen3 cross-env, both using Claude Opus 5 / HIGH.

The table reports Spearman ρ\rho (higher is better):

History ConditionOLMo3 in-envQwen3 cross-env
W00.620.74
Full history0.900.82
random160.760.72
embedding-top160.870.73
history-summary0.830.79

In OLMo3 in-env, embedding-top16 clearly outperforms random16 and approaches the full-history performance. In Qwen3 cross-env, its advantage over random selection is smaller, and its performance remains below the full-history result. This indicates the benefits of retrieval approach depend on the setting, and embedding similarity does not adequately identify the most valuable cross-env historical records. history-summary outperforms W0 in both settings but remains below full history. This supports the predictive value of summarization while also showing that this summary does not preserve all information in the full history.

W3: The influence of LLM's inner knowledge in cross-env and the use of agentic tools

Thank you for your comments. To further examine whether the results depend on Claude Opus 5's inner knowledge, we conduct additional cross-env prediction experiments with 6 model backbones using settings in Table 3.

The table below reports macro-average Spearman across 5 target envs.

BackboneRelease dateW0Four-source historyΔρ\Delta \rho
Qwen3-235B-A22B2025-04−0.050.20+0.25
DeepSeek-V3.22025-120.070.57+0.50
Mistral Large 32025-12−0.160.40+0.56
Kimi-K2.52026-010.100.58+0.48
GLM-52026-020.100.55+0.45
MiniMax-M2.52026-020.030.47+0.44

All 6 backbones improve predictions, indicating that cross-env transfer conclusion is not specific to Claude. In particular, Qwen3-235B was released in 2025-04, alongside Qwen3 and before Marin in 2025-05, Nanochat in 2025-10, and OLMo3 in 2025-11. Qwen3-235B therefore could not have encountered the target environments during training, making them genuinely novel research settings. Even in this setting, we observe an improvement of +0.25.

Use of agentic tools. RWM predictions were obtained solely through direct model API calls, without Claude Code or other agentic tools. All RWM predictions reported in the figures and tables were produced through this direct API workflow.

Dear Reviewer, thank you again for your review. We have posted responses with new analyses addressing your concerns Transfer beyond AutoRegressive Pretraining, Stronger LLM-based baselines, and the influence of LLM's inner knowledge in cross-env and the use of agentic tools. We would be grateful to know whether these address your concerns, and we are happy to clarify anything further before the discussion period ends.

Reviewer_EH6VOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a world model for automated research. The authors proposea method in which experimental environments in LLM research (e.g. models and computational resources) are provided as input to an LLM, which uses ICL to predict the outcomes of unseen experimental interventions.

Critical strengths

This paper demonstrates that LLMs can accurately predict the outcomes of unseen interventions in research environments, using large-scale experimental records from pretraining and post-training. In particular, the finding that experimental records improve the predictive accuracy of world models more substantially than the choice of LLM or its reasoning capability is interesting. The experimental dataset is also large-scale (170,000 GPU-hours), and the results are valuable because collecting such data is not easy.

Critical weaknesses
  • In a sense, it seems somewhat unsurprising that providing past experimental records in the context improves prediction accuracy. For example, in algorithm discovery research, it is already known that using experimental records through ICL can help select promising candidate programs for improvement. This can be regarded as a kind of world model. Therefore, the novelty of prediction using experimental records itself is limited, and I feel that the idea of LLMs serving as effective world models for automated research is not particularly novel.

  • The paper does not conduct input ablations to determine which information in the experimental records is necessary for prediction. It does not investigate which features or fields of the past experimental records provided in the context (e.g., model configurations, compute budgets, code, or improvements relative to the baseline) contribute to prediction accuracy. It also remains unclear whether it is necessary to provide the full records in a long context, or whether similar performance could be achieved using a small set of structured features.

Questions for authors
  1. Can the acquired knowledge transfer to substantially different model scales or training paradigms? How many experimental records are needed, and how does prediction accuracy change when the diversity of experimental records is varied?
  2. Which fields or features of experimental records are most effective for prediction?
4 comments

W1: Novelty and Empirical Contributions

We thank the reviewer for the insightful comments. We agree that prior work, including algorithm discovery with in-context experimental feedback, provides a foundation for using experimental feedback in research decisions, as discussed in Section 4 and Appendix A. In those settings, however, evaluations are typically cheap, so candidates can simply be executed, and history mainly guides search. Predicting outcomes before execution matters most when each experiment is expensive. This is the case in real LM development, where a single intervention costs hundreds of GPU-hours on average, and predicting outcomes in advance determines which experiments are worth running. This is the regime we study. Within this regime, our work makes three empirical contributions beyond these precedents.

(1) Cross-environment reuse of research knowledge. Prior work often evaluate usefulness of history within the same environment [2][3], which does not imply it can transfer when the environment changes. When taking Qwen3 as the target environment and providing only records from OLMo3, Nanochat, and Marin, our RWM prediction performance improves (Table 3). This shows that research knowledge can support predictions across model architectures and training recipes, extending its value beyond source environments.

(2) Numerical outcome prediction in expensive+complex AI training experiments (GPU-needed). Prior studies often adopt preference-based selection [1][2], which doesn't guarantee numerical performance prediction. In particular, [2] found numerical outcome prediction difficult in its initial experiments, further indicating establishing numerical predictive capability is not an obvious result. Different from prior work, RWM explicitly predicts numerical outcome, showing history can improve numerical predictions across broad action space. Our evaluation spans 9 research environments and over 2.6k experimental records, requiring over 171k H100 GPU-hours to construct. The interventions involving architecture and training recipe change, including attention scope and multi-token prediction (8 directions in total). The effects of these diverse interventions further depend on applied environments, which therefore adds complexity and requires empirical evaluation.

(3) The role of research knowledge relative to model capability and reasoning budget. Even if one expects records to help, the size of their effect relative to model capability is not obvious. The reviewer also notes this finding as interesting: experimental records improve the predictive accuracy of world models more substantially than the choice of LLM or its reasoning capability. This suggests that insufficient relevant research knowledge, rather than reasoning ability, is the main bottleneck for RWM prediction.

References

[1] Can We Predict Before Executing Machine Learning Agents?

[2] AI Research Preference Models

[3] Large Language Models to Enhance Bayesian Optimization.

W2: Input Field Ablations

We thank the reviewer for pointing this out. To clarify which historical information contributes, we conduct field ablations. The experiments cover the OLMo3 in-env and Qwen3 cross-env settings, both using Claude Opus 5 / HIGH.

W0 provides no history; “PRE-only” retains 5 fields: alias, reveal position, direction, mechanism labels, and intervention description; “PRE+gain” additionally includes the final gain; “intervention description+gain” retains only three fields: alias, complete intervention description, and gain; “full history” includes all 10 fields, including status, failure stage, loss jumps, and whether the final step was skipped.

The table below reports Spearman ρ\rho (higher is better):

History ConditionOLMo3 in-envQwen3 cross-env
W00.620.74
PRE-only0.680.72
PRE+gain0.900.81
Intervention description+gain0.900.83
Full history0.900.82

The final gain field provides clear incremental value in both settings. “Intervention description+gain” performs similarly to PRE+gain. This indicates that explicitly providing reveal positions, directions, and mechanism labels does not yield consistent gains. Full history does not show consistent improvements from the additional diagnostic fields. Therefore, under the evaluated settings, “intervention description+gain”(3 fields) preserves most of the predictive value of the full history.

Q1: Transfer across model scales and training paradigms, and the effects of experimental record quantity and diversity

Thank you for your questions. We added four experiments to examine the scope of transfer and the effects of experimental record quantity and diversity.

(1) Transfer across model scales. We use OLMo3-370M as the target environment and provide 305 experimental records from the 5 pretraining environments in the paper, without target-env history. We evaluate predictions on 145 target interventions (test set, cost additional ~29k H100 GPU hours).

History conditionMAE ↓\downarrowSpearman ↑\uparrow
No history5.85e-20.57
305 source-environment records5.29e-20.83

Providing source-environment records improves Spearman correlation from 0.57 to 0.83 and reduces MAE by approximately 9.6%. The target model is 3.70× the size of OLMo3-100M, providing additional evidence that pretraining experience can transfer to a larger model.

(2) Transfer across training paradigms and task types. See our response to reviewer HASF, ‘W1: Transfer beyond AutoRegressive Pretraining’

(3) Effect of experimental record quantity. See our response to reviewer jiT5, ‘W1 & Q2: ablation of experimental record quantity. ’

(4) Effect of experimental record diversity. We fix the size of history set at 61, keeping the Qwen3 target and the same test set of 71 interventions fixed. The 4 single-source conditions use records exclusively from OLMo3-100M, OLMo3-190M, Marin, or Nanochat, respectively. The multi-source conditions use 4 balanced assignments, each containing 15–16 records from each of the 4 environments.

History compositionMean MAE ↓\downarrowMean Spearman ↑\uparrow
Average across 4 single-source conditions9.65e-30.76
Average across 4 multi-source assignments8.46e-30.81

Multi-source history improves prediction accuracy relative to the single-source average, supporting the predictive value of experimental record diversity.

Q2: Most effective fields for prediction

See W2 for the field ablations.

Thank you for the authors' response! I find it interesting that experimental experience from smaller models can also help predict the training performance of a larger model. (I am personally interested in whether experience from models around 100M parameters can transfer to models with several billion to tens of billions of parameters. Even if such transfer does not work, I think clarifying its limitations would be valuable to the community.) Overall,I believe this work is valuable to the community, so I will raise my rating.

Thank you for your positive assessment! We agree that further investigating the scope and limits of transfer is valuable. Indeed, we are actively exploring transfer from smaller (<1B) to larger models (>7B) as one direction of our follow-up work on scaling up these evaluations. Beyond model scale, we are also investigating transfer across different domains and training paradigms in AI training.

Thank you again for recognizing the value of this work to the community. We hope to have the opportunity to share our follow-up findings with you and the broader community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.