- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This submission proposes a pipeline for generating new variations of environments (worlds) in the format of the DiscoverPhysics (Wiemann et al., 2026) framework.
The 'easy' worlds are constructed by varying several parameters of the DiscoverPhysics force laws. The 'hard' worlds are generated by prompting a frontier LLM (Claude Fable 5) to produce worlds in the style of DiscoverPhysics with some latent structure governing the dynamics; the prompt (given in full in Appendix F) instructs that each new world "must differ substantially" from the previous ones.
Then, the medium-difficulty worlds are generated using a simplified version of the above prompt, and kept if a base Qwen3-4B LLM attains an evaluation score in [0.4, 0.7]. For worlds outside this range, a frontier LLM (Claude Opus) receives a prompt containing the world's score and a "too hard" or "too easy" label, and revises the world or writes a new one. The resulting world is then tested with Claude Haiku 4.5 instead of Qwen3-4B, and kept if Haiku scores in [0.4, 0.7].
Finally, using the verl library, Qwen3-4B is fine-tuned with GRPO on the resulting worlds. The fine-tuned model is then evaluated on the hand-made DiscoverPhysics worlds, where it outperforms Claude Sonnet 4.6 on most metrics from the DiscoverPhysics work.
Critical strengths
-
Fine-tuned with GRPO, Qwen3-4B outperforms Claude Sonnet 4.6 on most DiscoverPhysics metrics when tested on the original DiscoverPhysics worlds. Fine-tuning removes most execution errors, such as crashing code and premature submissions, as shown in Figure 4.
-
The medium worlds are calibrated to a target difficulty: a frontier LLM (Claude Opus) revises or replaces each world using its measured score until a reference model scores it in a target band. This loop extends ideas from previous works, such as unsupervised environment design, to the problem of discovery of physics rules and provides a reusable way to fill the gap between easy and hard worlds.
-
'Hard' worlds are reimplemented by an LLM from their written description alone, with an adversarial test for cases where the two versions diverge. Unlike LLM environment generators such as OMNI-EPIC (Faldor et al., 2025), this is designed to reject worlds whose description underspecifies their code or whose numerics are unstable, and it gives others a reusable test for LLM-written simulators.
Critical weaknesses
-
Transfer beyond DiscoverPhysics is mixed, so the abstract's claim of improved performance on other scientific and reasoning benchmarks holds only in part. NewtonBench, which the paper ranks most similar to its training task, and HeroBench improve, but GravityBench is "slightly degraded", and GravityBench, HeroBench and LiveCodeBench scores peak at step 500 and fall by step 850 (Figure 5). Transfer is therefore benchmark-dependent, and for most benchmarks it does not grow with further training.
-
The easy training worlds overlap with the DiscoverPhysics test worlds, which the paper calls an "in-distribution test set": several easy worlds use the same force laws as public test worlds, differing only by a constant factor. So a large part of the benchmark prediction-error gain may come from familiar force laws rather than new discovery.
-
The results show that the full pipeline can help, but there is a lack of a systematic analysis of why various parts of the pipeline work and to what extent each of them is needed.
Overall, the submission appears to be a reasonable training setup, though perhaps not a significant advance. Its training loop combines existing parts, the DiscoverPhysics simulator and protocol and standard GRPO in the verl library. Its own contribution is the world-generation pipeline, whose value the experiments do not isolate sufficiently well.
Questions for authors
- How much of the headline drop in prediction error comes from a few worlds such as gravity? The largest gain in Figure 6 is on gravity, whose force law also appears in the easy training worlds, yet coulomb_easy, whose force law also appears there, gains only modestly, and several public and private worlds barely change. These gains seem far from the "uniform improvement to the normalised MSE for every world" that the main text reports.
1.1. Why is Figure 6 placed only in Appendix C, when Section 5.2 draws its per-world conclusions from it? Readers cannot easily check the main-text claims without leaving the main paper.
- Does the higher explanation score, one of the metrics on which fine-tuned Qwen3-4B beats Claude Sonnet 4.6, reflect better understanding or just wording that the scoring LLM favours? It is not easy to untuit this from the information I could see in the paper so far, but apologies if I missed something.
2 comments
Thank you for your response and for reporting the results of additional experiments. After taking these experimental results into account, I still have concerns about some of the central claims.
The messaging in the title and abstract implies that training on the generated worlds improves scientific discovery ability. The Conclusion states: "the RL training improved the scientific discovery ability of the Qwen model"; the Abstract states that the generated worlds "can provide effective training environments for experimentation and reasoning about physical phenomena" and claims "improved performance on other scientific and reasoning benchmarks."
The main results on DiscoverPhysics use an in-distribution test set, and the cross-benchmark gains are modest. Of the 11 public worlds, 7 overlap with the training templates. In the response to Reviewer fxys, the results on the remaining 4 non-overlapping worlds are now reported separately. While the results improve over the baseline in terms of trajectory fitting, a "pass" requires both low MSE and an explanation score of at least 0.75. None of the 20 attempts pass, and the mean explanation score only reaches 0.28. So, while the experiments show improvement over the baseline, they do not rise high enough so that one could claim successful discovery of unfamiliar laws. Claims about "Scientific Discovery" would need support from strong performance in out-of-distribution cases. Otherwise, the improvement can be attributed to better trajectory fitting, not to law discovery.
Response and extra ablations
Firstly, we do agree with the reviewer that the cross-benchmark improvements are modest in most cases. We would like to point out that we believe it is of interest to the community to show if improving scientific discovery ability (as measured by DiscoverPhysics) correlates with improvements on other tasks.
Q1: We also agree the importance of the LLM-generated worlds was not adequately shown, hence we performed a direct comparison to the same Qwen3-4B model trained on only the template worlds, showing the results below.
Figure here: https://postimg.cc/Wt60MwDd
If the gains came from familiar force laws, training only on the 78 template worlds, which contain all of those force laws, should reproduce them. It does not according to the geometric-mean MSE is 0.59 vs 0.085 for the full pool, mirrored in the fit pass rate halving, and the template-only run plateaus without reaching the pass threshold. The automated generation pipeline is therefore the component that drives the trajectory-accuracy gain, even for an identical underlying model. We show a breakdown of the performance on the worlds in the evaluation set that crossover with template worlds, compared to those that do not in the figure below. We see gains across all world types, especially when using the full template + self-generated world training set.
Figure here: https://postimg.cc/xcdZbCsn
We also point to the analysis done in the response to Q3 of Reviewer fxys for further evidence that performance increases on DiscoverPhysics worlds that are not similar to the laws seen in the template worlds.
Q1.1: We also agree that Fig. 6 is useful in the claims for Section 5.2, and we will move the figure there from the appendix accordingly. This makes it more clear that we see gains in the MSE fit to 21/22 worlds, which we also further clarify in the text.
Q2: To test this, we perform a judge ablation where we rescore each world’s explanation with three non-Claude judges. We see the three alternate LLM-judges give improvements to the explanation score of equal to, or more than the improvement on the Claude Sonnet judge, so we do believe the RL training has resulted in greater conceptual understanding of these worlds.
Figure here: https://postimg.cc/BPqLhR8Q
We also do a “key word” sensitivity test where we scored explanations for 4 benchmark worlds (yukawa, extra_dimensions, coulomb_easy, and one private world) under four conditions, with all four judges and 3 samples per judge:
Every judge scores the correct-but-unnamed physics (B) a near perfect score, and the named-but-wrong physics (A) near the bottom well below a pass. So the score is driven by explanation accuracy rather than terminology. Appending a sentence of rubric keywords to real failed explanations (D) gives limited partial credit (+0.04 to +0.10), and padded explanations remain far below correct ones.