acceptodds
Under review as a conference paper at ICLR 2027

RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement

Abstract

Recursive self-improvement requires turning evidence of model failures into better models. Data-centric post-training research entails diagnosing capability gaps, designing and validating training-data strategies, and learning from checkpoint feedback. Can LLM agents automate this loop? Existing benchmarks entangle research decisions with optimization, serving, evaluation, and systems implementation, obscuring agents' research capability. We introduce RSIBench-Data, a controlled benchmark of LLM agents as data-centric researchers with a fixed post-training stack. Agents iteratively revise training-data strategies for a fixed target model; training and serving use Tinker-backed services, official evaluation runs through Harbor and E2B sandboxes, and budgets are fixed across agents. We evaluate four frontier agents on six benchmarks across software engineering, terminal use, scientific question answering, and mathematics. Agents demonstrate core data-centric research capabilities: in 58.33% of settings, they improve upon the first valid attempt by refining strategies from feedback. However, improvement is inconsistent. Among searches continuing after the best observed score, 78.26% end with a lower-scoring final attempt, while the rest only recover the same peak. A strong candidate may therefore appear early or midway through a run even as later revisions fail. Trajectory analysis identifies four patterns in stronger runs: accurate hypotheses, validation-grounded supervision, behavior-aligned data, and preservation of strong checkpoints. These findings suggest that current agents can make useful data-centric discoveries but cannot yet translate feedback into consistent improvements. RSIBench-Data provides a measurable, auditable testbed for the research capabilities required for recursive self-improvement.

Primary area:
datasets and benchmarks
Keywords:
RSI, Benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EzPgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RSIBench-Data is a benchmark that tests whether LLM agents can do data-centric post-training research: find where a model is weak, design training data to fix it, and revise that data based on feedback. Everything except the data is fixed. The authors tested 4 agents on 6 benchmarks. The main finding is a "discovery–reliability gap”.

Critical strengths
  1. The topic is important now; we need to judge the model's ability like RSI

  2. The analysis is interesting in Section 4 and the appendix.

Critical weaknesses
  1. The idea is quite similar to DatasetResearch and InnovatorBench. (Just different tasks)

  2. There is a risk of reward hacking. (Search data online? Just modify the test set a little bit?)

  3. The scope is small.

Questions for authors
  1. What did the agent really do? Sourcing data online or something else? Can the agent just download something online?

I'm really concerned about whether the score reflects ability or is just reward hacking. I may raise my score if my concern is solved.

Reviewer_xKjkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RSIBench-Data, a benchmark in which agents iteratively construct training data and select an improved checkpoint of a fixed target model. It evaluates four agent systems on six benchmarks and reports that agents often find better candidates but struggle to maintain their best performance.

Critical strengths

S1: The problem is relevant: can agents turn model failures into useful training data? A shared training and evaluation pipeline provides a useful starting point for studying this question.

S2: The paper records datasets, configurations, costs, and checkpoint scores. The concrete case studies could help others understand how agents revise their data strategies.

Critical weaknesses

W1: The connection to recursive self-improvement and the benchmark's added contribution are not sufficiently established. The main experiments use a fixed strong teacher, Opus 4.8, to generate trajectories for a separate, weaker target model. This demonstrates agent-directed post-training, but not a recursive loop in which the improved model contributes to subsequent improvement. Although the paper positions data-centric research as a component of RSI, it does not show how performance here translates into that capability. Since the benchmark combines existing tasks and services, the paper should also clarify what new evaluation capability the protocol provides beyond their integration.

W2: The experimental comparisons are insufficient to establish the benefit and reliability of agent-driven data research. Simple non-agent baselines, such as seed-data SFT and one-shot teacher distillation with fixed configurations, are missing. Since the target is a base model, some gains may reflect learning tool protocols and output formats, as suggested by the task-completion behaviors in Appendix B.6. Using instructed model would help separate such adaptation from broader capability gains. Moreover, each agent–benchmark setting has only one reported representative run (Section 5); repeated training and evaluation are needed to establish the stability of the improvements.

W3: The official evaluation is not an independent test. Checkpoint selection and final evaluation use the same task subset (Section 5), while agents receive task-level trajectories and verifier feedback during selection (Section 2.4). The authors acknowledge this limitation, but rerunning these tasks in fresh environments does not establish generalization to unseen tasks. The prohibition on using evaluation materials as supervision (Section 2.3) also does not eliminate adaptation to evaluation feedback. A private held-out test split and training–evaluation overlap checks are needed to assess generalization and clarify the scope of the “Train/Eval Data Isolation” claim in Table 3.

Questions for authors

Q1: What evidence connects performance in this fixed-teacher post-training setup to recursive self-improvement? What can the proposed protocol evaluate that existing data-curation or post-training benchmarks cannot?

Q2: How much improvement remains over fixed-configuration seed-data SFT and one-shot teacher distillation under comparable resource budgets? Does it persist with an instruction-tuned target and across repeated training and evaluation runs? Please report the variation across runs.

Q3: Can you evaluate the selected checkpoints on a private task split never exposed through selection feedback and report overlap between submitted training data and evaluation tasks? How is the data-isolation claim in Table 3 enforced, and what does it guarantee given the reuse of selection tasks?

Reviewer_WnXoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a benchmark to evaluate Research Agents on data-centric tasks particularly those related to post-training data for LLMS.

Critical strengths

While I think the domain that this paper looks at is relevant and important I could not find significant strengths to justify acceptance.

Critical weaknesses
  1. Unclear how this benchmark stands alone amongst other benchmarks - the paper presents a benchmark that seeks to isolate a very specific step in post-training research. While it is important to control for variables, especially in systems with many parts that can change and affect the outcome such as research agents, the authors would benefit from a stronger motivation why this subtask raises to such a level that it warrants its own benchmark rather than existing as part of a larger comprehensive research agent benchmark.

  2. Insufficient analysis/synthesis of findings - The benefit of isolating a specific subtask and controlling for other variables is the ability to identify more specific strategies that improve the agent performance. I recognize that some of this is in the appendix but I would dedicate more time to analysis in the main body of the paper. This analysis will help to justify why a benchmark such as this is useful beyond just improving agent performance on other existing benchmarks.

  3. Presentation of findings is unclear without further investigation - there is only one table of findings in the main body of the paper so it carries a lot of weight. The authors would benefit from making that table clearer what is being done. Because it is common for the models in that table to be benchmarked as agents (and in reality have much higher scores by themselves as agent) when looking at this table it is confusing that the models are listed as it seems to be benchmarking those specific models rather than their data centric capabilities. Similar to point 2 I would try to break down the analysis more into a few different formats to unpack what learnings this benchmark can offer to researchers.

Questions for authors
  1. Is 'data synthesis' the extent of data-centric AI? Are there other aspects to it that need to be studied?

  2. Are there other agent benchmarks that would allow you to isolate the effect the of data selection/synthesis? How does your set up compare?

  3. Data-centric self-improvement implies that the model is selecting its own training data. In the case of this benchmark you're studying the ability of other agents to select training data for a weaker model. Did you look at taking the base model and seeing how far it could improve itself with just modifications to its own training data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.