acceptodds
Under review as a conference paper at ICLR 2027

POSEIDON: Scaling Training Environments for Scientific Agents from Code Repositories

Abstract

Training environments remain a major bottleneck for agents in data-driven scientific discovery. Existing collections of executable and verifiable training environments are small, and their construction remains difficult to scale. To address this limitation, we introduce POSEIDON, an automated pipeline for generating diverse, verifiable training environments from scientific code repositories at scale. From the same repository budget, POSEIDON extracts nearly 10× as many training environments as a prior approach by decomposing scientific workflows into four complementary task types: Reproduction, Subflow, Completion, and Correction, where each type targets a distinct capability for scientific research. A recursive improvement loop further refines failed candidates and recovers otherwise discarded environments. Using POSEIDON, we construct PEARL, a collection of 97,659 executable and verifiable environments from 7,110 public repositories spanning 10 scientific disciplines. To the best of our knowledge, PEARL is the largest and most broadly multidisciplinary collection of scientific-agent training environments to date. Qwen3.5 models (9B, 27B, and 35B-A3B) trained on PEARL achieve consistent gains across six challenging scientific-agent benchmarks. Overall, the scale and diversity of PEARL offer opportunities for training scientific discovery agents. Code is available at https://anonymous.4open.science/status/POSEIDON-89D7.

Primary area:
foundation or frontier models, including LLMs
Keywords:
llm agent for scientific discovery, data synthesis method

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_c2U5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes POSEIDON as a pipeline for generating training environments from code repositories. The authors also constructed PEARL, a collection of such environments. Experiments showed that training on PEARL improved Qwen 3.5 models performance on six scientific-agent benchmarks.

Critical strengths
  1. POSEIDON extracts 10 times more training environments than a prior approach, making more efficient use of existing code repositories.
  2. PEARL, as the largest collection, contributes to the community and the scientific agent field.
  3. Manual review ensures task quality, although the paper only sampled 600 tasks for review.
Critical weaknesses
  1. The code link provided https://anonymous.4open.science/status/POSEIDON-89D7 is incorrect and contains no content. It should be https://anonymous.4open.science/r/POSEIDON-89D7. Moreover, the source commit modification date is October 8, 2026, many days after the submission deadline. This is not good practice and unfair to other timely submissions.
  2. Distribution of the collected scientific repositories across the disciplines are very unbalanced, with biology accounting for too high a percentage at 47%. Is a more balanced distribution possible? The authors mention oversampled the biology repository because of its active development, but 47% is still too high.
  3. The manual review acceptance rate is 95.5% for 600 tasks sampled. Ideally, all environments should undergo manual review. Since the acceptance rate isn't 100%, it means some aspects are problematic. However, I understand this would be very costly. Another approach might be to summarize the remaining 0.5% and identify the problems, then extract common issues and perform model-assisted filtering on all environments.
Questions for authors

Please address the weaknesses above. Figure 3: The title is too short, without information for components a, b, and c.

7 comments
AuthorsOpenReview ↗

Response to Reviewer c2U5 (Part1/3)

Correction of the Code Link and Clarification of Release Updates (W1)

Thank you for pointing out the incorrect link and raising your concern about the timing of the code update. We sincerely apologize for providing the repository status page instead of the code-browsing page. The correct link is https://anonymous.4open.science/r/POSEIDON-89D7. We have corrected the code link on the first page of the revised manuscript.

The October 8 update was intended to address omissions and outdated components in the public code release. We updated the qc directory, which contains the quality-control workflow and the Docker image build-and-push workflow. We also removed outdated code for two task types explored during early development, retaining the four task types described in the paper.

These updates were intended to improve the completeness and consistency of the public release. Nevertheless, we recognize that we should have completed this preparation earlier. We sincerely apologize for the delayed update and the confusion and additional review effort it caused. Thank you for bringing these issues to our attention.

Training with a More Balanced Disciplinary Distribution (W2)

Thank you for highlighting the disciplinary imbalance. To investigate whether a more balanced training distribution can retain the benefits of our environments, we constructed a smaller SFT dataset by downsampling biology and pharmacology trajectories from the original training collection. The resulting dataset contains 36,721 trajectories, approximately 57% of the original 64,477 trajectories, with no single discipline accounting for more than 22.2%.

DisciplineFinal CountProportion
Biology8,16322.2%
Pharmacology8,04121.9%
Physics5,63115.3%
Environmental Science3,94210.7%
Engineering3,1408.6%
Materials Science2,0165.5%
Mathematics1,9845.4%
Astronomy1,4844.0%
Chemistry1,4113.8%
Earth Science9092.5%
Total36,721100%

We then fine-tuned Qwen3.5-35B-A3B on this rebalanced dataset. The results are shown below.

ModelBioCoderMA-BenchSA-BenchBixBench MCQBixBench OpenSciCode MainSciCode SubBioMysteryAvg
Qwen3.5-35B-A3B54.571.338.257.132.213.837.527.841.6
+ SFT (original distribution)60.478.347.169.337.120.045.128.948.3
+ SFT (rebalanced distribution)56.574.042.270.238.013.843.826.745.7

Despite using approximately 43% fewer trajectories, the rebalanced dataset improves the average score from 41.6 to 45.7, with gains on six of the eight evaluation metrics and comparable performance on the remaining two. Although the rebalanced subset achieves a lower average score than the full dataset, it retains clear overall gains over the untuned backbone despite using substantially fewer trajectories. The performance difference may reflect the combined effects of reduced data quantity and the adjusted disciplinary composition.

These results demonstrate that a more balanced training mixture is feasible and retains training value. The benefits of our environments are therefore not restricted to the original disciplinary distribution.

We have added this experiment to Appendix H, with the rebalanced data distribution in Table 10 and the evaluation results in Table 11.

AuthorsOpenReview ↗

Response to Reviewer c2U5 (Part2/3)

Analysis of Human Audit Findings and Targeted Filtering (W3)

Thank you for this constructive suggestion. We reviewed the cases flagged during the human audit and found that most involved isolated presentation issues that did not affect task solving or evaluation, such as inconsistent capitalization, redundant instructions, and overly detailed background descriptions. Two tasks, however, shared a terminology issue: the scientific metric names did not precisely match the computations specified in the instructions. A representative example is provided below.

Task

You are provided with data/validation_coords.pt, a dictionary containing
three NumPy arrays:
- p1: predicted coordinates of shape
  (num_samples, num_residues, num_atoms, 3)
- p2: true/experimental coordinates of the same shape
- mask: a boolean array of shape (num_samples, num_residues), where
  1 indicates a valid residue and 0 an invalid residue

Your goal is to evaluate the accuracy of the predicted protein coordinates
against the experimental structures using Global Distance Test (GDT)
scores. Do not perform any structural superposition or alignment; use
the coordinates exactly as provided. Use only the C-alpha atom of each
residue (atom index 0 along the atom axis).

For each sample, compute:
- The Euclidean distance between the predicted and true C-alpha
  coordinates for each residue.
- Restrict evaluation to residues marked valid by mask; only valid
  residues contribute to the numerator and denominator.
- For a given distance cutoff c, the fractional score is:
  (# valid residues with distance <= c) / (# valid residues).
- GDT_TS is the average of these fractional scores over cutoffs
  [1, 2, 4, 8] Å.
- GDT_HA is the average of these fractional scores over cutoffs
  [0.5, 1, 2, 4] Å.

Compute these per-sample scores for the whole validation set, then
aggregate them across samples. Report four aggregate values:
mean GDT_TS, median GDT_TS, mean GDT_HA, and median GDT_HA.

Issue

The task contains a terminology mismatch. Standard GDT involves searching
for structural superpositions to evaluate the fraction of corresponding
residues within specified distance thresholds. This task instead computes
distances directly in the supplied coordinate frames, making the scores
sensitive to overall rotation and translation. Although the requested
computation is explicitly specified, calling these quantities GDT_TS and
GDT_HA without qualification is potentially misleading. More precise names
would be "raw-frame distance-threshold scores" or "GDT-like scores without
superposition."

We translated this recurring issue into a targeted model-assisted screening criterion and applied the following prompt across the full collection:

You are a scientific terminology reviewer.

Check whether the scientific names used in the following task accurately
describe the computations defined in the task itself. Identify names that
are misleading because the stated computation omits or changes an essential
part of their conventional definitions.

Return only valid JSON:

{
  "has_misleading_names": true,
  "issues": [
    {
      "name": "",
      "mismatch": "",
      "suggested_name": ""
    }
  ]
}

If no misleading names are found, set "has_misleading_names" to false
and "issues" to [].

Task:
{prompt}

This screening flagged 127 environments, approximately 0.13% of the full collection. We removed these environments and their associated training trajectories.

Human review before large-scale generation. Human auditing was also conducted during pilot generation, before we scaled up production. These early reviews identified more substantial recurring problems, including inconsistent data-path prefixes (data/ versus data/data/), graders incorrectly accepting invalid or non-answer outputs such as NaN, None, empty lists, and empty dictionaries, and tasks unrelated to scientific reasoning, such as scraping paper-author information or troubleshooting GPU memory capacity. We addressed these issues before proceeding with large-scale generation.

Thus, human review served both to improve the generation pipeline before scaling and to identify residual issues for targeted filtering afterward.

AuthorsOpenReview ↗

Response to Reviewer c2U5 (Part3/3)

Expanded Caption for Figure 3 (Q1)

We have expanded the caption of Figure 3 to provide clearer descriptions of components a, b, and c.

The detailed caption now is:

Figure 3: Overview of POSEIDON. (a) Repo Discovery: expands subject keywords with an LLM, searches GitHub for repositories meeting our requirements on stars, data, and paper availability, and builds an executable sandbox for each. (b) Task Synthesis: groups a repository’s code files by scientific workflow and derives four task types from them: Reproduction, Subflow, Completion, and Correction. (c) Verification: a generator–validator loop with recursive feedback ensures environment reliability and scientific validity.

Reviewer_tccuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

POSEIDON converts scientific code repositories into executable training environments through workflow extraction, task generation, verification, and repair. Its PEARL collection contains 97,659 environments from 7,110 repositories. Training on 64,477 successful trajectories improves three Qwen backbones across a suite of six scientific-agent benchmarks.

Critical strengths
  1. The pipeline produces a substantial training resource spanning multiple scientific disciplines and task types. The 600-environment human audit, including execution and evaluator checks, provides meaningful support for the quality of the generated tasks.

  2. Transfer gains are demonstrated across three backbone sizes and several external benchmarks. Comparisons at similar trajectory counts and ablations of repository diversity and task composition go beyond demonstrating that more training data helps.

Critical weaknesses
  1. The claimed comparison under a similar data budget matches trajectory counts rather than training tokens. Table 6 reports average response lengths of 14,262 tokens for PEARL, versus 10,073 for D3-Gym and 2,393 for MedAgentGym. Table 4 therefore does not isolate data quality or diversity from the amount of supervision.

  2. Repository-level contamination filtering is described, but its matching criteria and removal counts are not reported. Since both training tasks and evaluations draw on public scientific code, the independence of the transfer results remains difficult to audit.

Questions for authors
  1. Does PEARL retain its advantage when total training tokens and optimization budgets are matched across datasets?
4 comments
AuthorsOpenReview ↗

Response to Reviewer tccu

Comparison under Matched Token Budgets and Optimization Settings (W1, Q1)

Thank you for suggesting a more rigorous experimental comparison. To strengthen the comparison, we conducted additional experiments controlling both the dataset token budget and shared training hyperparameters.

  1. Matched dataset token budgets. We downsampled PEARL trajectories and upsampled MedAgentGym trajectories to match the token budget. The resulting datasets contain 19,091,807, 19,085,943, and 19,084,602 tokens for PEARL, MedAgentGym, and D3-Gym, respectively, with a maximum difference of less than 0.04%.

  2. Matched optimization settings. We used the same training configuration for all three datasets: a global batch size of 128, a learning rate of 5e-6, a cutoff_len of 32,768, and 3 epochs.

Model# TokensBioCoderSciCode MainSciCode SubMA-BenchSA-BenchBixBench MCQBixBench OpenBioMysteryAvg
Qwen3.5-27B–57.112.342.775.734.361.534.136.744.3
+ MedAgentGym19,085,94361.0 (+3.9)13.8 (+1.5)44.1 (+1.4)81.0 (+5.3)36.2 (+1.9)53.7 (-7.8)32.7 (-1.4)30.0 (-6.7)44.1 (-0.2)
+ D3-Gym19,084,60259.1 (+2.0)10.8 (-1.5)38.9 (-3.8)73.7 (-2.0)36.2 (+1.9)62.0 (+0.5)25.9 (-8.2)28.9 (-7.8)41.9 (-2.4)
+ PEARL19,091,80761.7 (+4.6)12.3 (+0.0)43.1 (+0.4)78.3 (+2.6)41.2 (+6.9)62.4 (+0.9)35.6 (+1.5)33.3 (-3.4)46.0 (+1.7)

Under these controlled settings, PEARL achieves performance comparable to or slightly better than the trained baselines, with an average score of 46.0 versus 44.1 for MedAgentGym and 41.9 for D3-Gym. These results support the effectiveness of our training data under closely matched dataset token budgets and identical optimization settings. Moreover, POSEIDON’s automated pipeline is designed to support efficient scaling of training data, offering greater scope for further improvements through larger-scale training. We have added the experimental settings and results of this comparison to Appendix G and Table 9.

Repository-level contamination filtering is described, but its matching criteria and removal counts are not reported. Since both training tasks and evaluations draw on public scientific code, the independence of the transfer results remains difficult to audit.

Contamination Filtering: Criteria and Removal Counts (W2)

We apply contamination filtering at two stages. First, during repository collection, we exclude the 40 source repositories explicitly identified in the public metadata of BioCoder and ScienceAgentBench (10 from BioCoder and 30 from ScienceAgentBench). This prevents these repositories from being used to construct our training environments.

Second, after collecting the distilled trajectories, we apply n-gram-based contamination filtering with n=13n=13, comparing the collected trajectories against the evaluation benchmark questions. Trajectories flagged by this procedure are removed from the training set, resulting in the exclusion of 1.3% of the collected trajectories.

We have clarified this two-stage decontamination procedure in the Training Details paragraph of Section 4.1.

AuthorsOpenReview ↗

Kind Reminder for Discussion

Dear Reviewer tccu,

As the discussion period draws to a close, we are very interested in hearing any further feedback you might have and would be pleased to clarify or address any remaining concerns. Please do not hesitate to let us know if you have additional comments or questions.

Thank you again for your time and thoughtful review.

Best regards, Authors of Paper 38687

Thank you for the detailed response and additional experiments. The matched-token comparison substantially addresses my main concern, and the clarification of the contamination filtering procedure is helpful. Overall, my concerns have been largely addressed, and I will maintain my positive rating.

AuthorsOpenReview ↗

Sincere Thanks

Thank you for your thoughtful review and feedback! We are glad that the additional experiments and clarifications have largely addressed your concerns. We sincerely appreciate your time and engagement throughout the discussion.

Reviewer_MYY5OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper turns the github repositories corresponding to scientific codes into more than 97k runnable tasks, each with data, a reference answer and automatic grader. each workflow is split into 4 task types such as run all of the task, part of it, fill missing part and fix planted bug. They show a model trained on this large scale scientific tasks performs well compared to the baseline on trained model on several other benchmark.

Critical strengths
  • A new, useful, large-scale benchmark to train and evaluate models is generated and released, which is a meaningful contribution.

  • The human audit is done carefully and the work considers the SOTA and its distinction with past works like D3-Gym clearly.

Critical weaknesses
  • the title says training environment but training is done only via SFT and the grader is only used for rejection sampling and there is no RL

  • table 2 results compares only against untrained model vs full PEARL. But when comparing against other trained baselines as in table 4, the comparison is limited the very small size of finetuning

  • The comparison in Table 4 is not fair since the teacher model involved in generating D3-Gym is based on the Qwen3 model, while PEARL is generated via DeepSeek V4 Pro, which is a much stronger teacher. A fair comparison would regenerate D3-Gym using the same model as the basis for fine-tuning.

  • The automated grader is not really fully tested and evaluated. Given that all the training is done using SFT and no RLVR is reported, the utility of the automated grader is not tested thoroughly and is only validated partially by llm check (App B).

Questions for authors

see weakness section

7 comments
AuthorsOpenReview ↗

Response to Reviewer MYY5 (Part1/2)

Reinforcement Learning beyond SFT (W1)

Thank you for highlighting the importance of RL experiments in validating the utility of our training environments. Due to time and computational resource constraints, our original submission included only SFT experiments. To address this concern, we have now conducted additional RL experiments to evaluate whether our environments support further improvements beyond SFT.

Experimental setup. Of the 97,659 training environments generated by POSEIDON, 64,477 were used for SFT. We sample candidate tasks from the remaining environments and assess their learnability by generating four rollouts per task using the SFT checkpoint of Qwen3.5-35B-A3B. We retain only tasks with at least one successful and one unsuccessful rollout, continuing this process until 3,000 training environments have been collected for RL. Starting from the same SFT checkpoint, we perform GRPO training using outcome-based rewards provided by our grader, with 200 training steps, a batch size of 256, a learning rate of 1e-6, and a maximum context length of 131,072 tokens.

ModelBioCoderMA-BenchSA-BenchBixBench MCQBixBench OpenSciCode MainSciCode SubBioMysteryAvg
Qwen3.5-35B-A3B54.571.338.257.132.213.837.527.841.6
+ SFT60.478.347.169.337.120.045.128.948.3
+ RL62.381.349.074.139.520.046.231.150.4

Results. RL further improves the average score from 48.3 to 50.4, yielding an additional gain of 2.1 points over SFT.The largest gains are on BixBench MCQ (+4.8 points) and MA-Bench (+3.0 points). These results demonstrate that POSEIDON’s training environments support further performance gains through RL beyond those achieved by SFT, and that our grader can provide effective reward signals for policy optimization in addition to supporting rejection sampling. We have added the RL experimental setup and results to Section 4.4 and Table 6 of the revised manuscript.

Rationale for the Small-Scale Baseline Comparison (W2)

The small fine-tuning dataset used in Table 4 was chosen to ensure a fair comparison with the trained baselines under comparable training data budgets. Specifically, D3Gym generated only 2,153 trajectories for fine-tuning. We therefore sampled 2,000 trajectories from our dataset for SFT, rather than using the full dataset. This comparison evaluates the effectiveness of our training data at a comparable scale, reducing the influence of differences in training data quantity.

Comparison Using the Same Teacher Model (W3)

Thank you for highlighting this important point for a more rigorous comparison. To control for the teacher model, we used DeepSeek-V4-Pro to generate rollouts in D3-Gym environments, collecting 2,200 distilled trajectories, available at https://anonymous.4open.science/r/distill_dsv4pro_d3gym-F132/. We fine-tuned Qwen3.5-27B on these trajectories and compared it with the same backbone fine-tuned on 2,000 trajectories generated by DeepSeek-V4-Pro in our PEARL environments. The results are shown below.

Model# TrajectoriesBioCoderSciCode MainSciCode SubMA-BenchSA-BenchBixBench MCQBixBench OpenBioMysteryAvg
Qwen3.5-27B–57.112.342.775.734.361.534.136.744.3
+ D3-Gym2,20061.0 (+3.9)12.3 (+0.0)43.8 (+1.1)76.3 (+0.6)41.2 (+6.9)63.4 (+1.9)28.3 (-5.8)31.1 (-5.6)44.7 (+0.4)
+ PEARL2,00062.3 (+5.2)13.8 (+1.5)43.4 (+0.7)77.7 (+2.0)38.1 (+3.8)63.9 (+2.4)37.6 (+3.5)34.4 (-2.3)46.4 (+2.1)

With both datasets distilled from the same strong teacher model, PEARL remains comparable to or slightly better than D3-Gym, achieving an average score of 46.4 versus 44.7 while using fewer trajectories. Beyond this comparison, POSEIDON’s automated environment-generation pipeline supports scaling to a much larger training collection, offering scope for further performance improvements through larger-scale training. We have added this teacher-controlled comparison to Appendix G, with the results reported in Table 8.

AuthorsOpenReview ↗

Response to Reviewer MYY5 (Part2/2)

Validation and Practical Utility of the Automated Grader (W4)

We would like to clarify that our grader validation extends beyond the LLM-based checks described in Appendix B. It combines programmatic testing, LLM-based verification, and human auditing. Our newly added RL experiments further assess the grader’s utility as a reward mechanism.

1. Programmatic testing and LLM-based verification. During environment generation, each grader first undergoes programmatic tests. Specifically, we verify that it accepts the task’s gold-standard result and rejects invalid or trivial outputs, such as None, NaN, infinity, empty lists, and empty dictionaries, where these do not constitute valid answers to the task. The surviving environments then undergo LLM-based verification to further assess consistency between the task requirements and the grading logic.

2. Human auditing before and after large-scale generation. Before scaling up generation, we conducted a small pilot and manually reviewed the resulting environments. This review identified some common grading issues, including NaN values bypassing numerical checks, non-integer counts being accepted after truncation through int() conversion, and incomplete validation of multi-component answers. We addressed these issues before proceeding with large-scale generation. After generation, we additionally conducted an independent human audit of 600 training environments, achieving an overall acceptance rate of 95.5%, and the issues identified in the remaining cases were minor, such as inconsistent variable naming, inconsistent capitalization, and imprecise scientific terminology, and were unrelated to the grader.

3. Evidence from downstream training. The grader supports rejection sampling during SFT and provides outcome-based rewards during RL. Both stages yield improvements on downstream evaluations. In particular, the newly added RL experiments improve the average score from 48.3 to 50.4 over the SFT checkpoint. These results provide complementary evidence that the grader supplies useful signals for both trajectory selection and policy optimization.

Together, these checks and training results support the grader’s reliability and practical utility beyond LLM-based verification alone. We have expanded the description of programmatic grader validation in Section 3.2.2 and added evidence of its utility for RL in Section 4.4.

Thank you for the detailed response and the additional experiments. The RL results and the teacher-controlled comparison with D3-Gym address my questions, and I appreciate the effort. However, I will keep my score for now. With the same teacher, the margin over D3-Gym is small (46.4 vs 44.7), the RL gain over SFT is about 2 points, and all results come from single runs on benchmarks with about 100 items or fewer, where a few questions can change the average. Since the main contribution of the paper is scale, I would like to see results across several training sizes (e.g., 2k, 8k, 32k and 64k trajectories) with multiple seeds, showing that the gains grow with scale and clearly exceed seed variance. Results like these would change my assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Summary

Dear Reviewers,

Thank you for your thoughtful and constructive feedback. We have revised the manuscript to address your comments, with the following major updates:

  • We added Section 4.4, “RL Potential,” demonstrating that GRPO training on our environments yields additional performance gains beyond SFT.
  • We added Appendix G with more controlled baseline comparisons, including a comparison using the same teacher model and experiments with closely matched dataset token budgets.
  • We added Appendix H to evaluate training with a more balanced disciplinary distribution, showing that the rebalanced subset retains benefits over the untuned backbone.
  • We revised the manuscript and added further methodological details, including expanded descriptions of components a, b, and c in the Figure 3 caption, clarification of the two-stage benchmark decontamination procedure, and additional details on grader validation.

We appreciate the reviewers’ suggestions, which have helped us improve the rigor and clarity of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.