acceptodds
Withdrawn from ICLR 2027

SPhyR: Environment for Spatial-Physical Reasoning with Simulation Verifier

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reinforcement learning from verifiable rewards needs verifiers, and the available ones are narrow: exact match, unit tests, or a learned reward model. Each assumes a task has a checkable canonical answer. Many design problems do not. We present SPhyR, an environment for spatial-physical reasoning whose verifier is a physics simulation rather than a comparison to a stored answer. An episode presents a 2D domain with prescribed loads and supports and a masked material distribution; the agent submits a completion, and the environment solves it as a linear elastic structure and scores it against a topology optimizer re-run on exactly the masked region. A completion that routes the load differently from the reference but just as efficiently scores just as well. Across the valid completions in our results, carry their load correctly while differing from the reference somewhere; a string-matching verifier rejects them all, and graph connectivity disagrees with compliance on of them, so a cheaper structural proxy will not substitute. We show the two views rank models differently and in both directions, that the environment's reward is not reachable by retrieval, and that a benchmark of this shape can rank models by grid-serialisation ability rather than by reasoning unless that is measured separately. The environment is released in the OpenEnv interface, single-step or with simulator feedback between attempts (Footnote: The dataset, environment and evaluation code are released; links are withheld for double-blind review and the code accompanies the submission as supplementary material.).

Primary area:
foundation or frontier models, including LLMs
Keywords:
verifiable rewards, physics simulation verifier, spatial reasoning, physical reasoning, topology optimization, benchmark, environment, OpenEnv, large language models, structural design, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_48DEOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SPhyR recasts 2D topology-optimisation as a grid-completion task and grades outputs against a physics simulation rather than a stored answer, which captures what cell-matching misses. The paper releases SPhyR as an OpenEnv environment with optional simulator feedback between attempts, which helps a model to revise its answers. It evaluates eleven models zero-shot on the v1-dataset and three models on the v2 dataset, demonstrating the advantage of physics based verifer rather than cell matching.

Critical strengths
  • The verifier is well-suited to a task with many valid answers. Figure 1 shows the limitations of using cell-matching to grade answers. Section 7 shows exact match is zero for every model while load-carrying varies from 33-99%, so a verifier is needed to tell models apart.
  • Section 8 finds samples in v1 are close together and leave 20% of cells visible, which the authors proceed to fix. The authors also identify that scores large-grid is partly explained by a model's ability to format correctly on SPhyR-v2, which is then accounted for and reported in the evaluation metrics. Both examples show rigorous and transparent auditing of the benchmark.
Critical weaknesses
  • SPhyR's entire framing is around its use as an environment for reinforcement with verifiable rewards yet no policy is ever trained against its reward. Without any training experiments the paper cannot answer if the reward is learnable or hackable as well as several other properties which are critical to assess the utility of a RL environment. This is a critical gap of the paper and would require a full set of training experiments to address. In the absence of training SPhyR is an evaluation benchmark with a sophisticated verifier.
  • SPhyR is a single task environment focused on topology optimisation where there are only a few difficulty settings. As a benchmark a single environment provides narrow insight into a model's spatial-physical reasoning capability. As a training environment, it supplies one narrow skill whose utility rests on whether skill transfers to other spatial-physical tasks, which SPhyR does not test.
Questions for authors

Please can you conduct some RL training experiments with an appropriately sized model e.g. 4B-8B parameters?

1 comment

Dear Reviewer 48DE,

Thank you very much for your time and effort in reviewing our work; this is highly appreciated. We would like to respond to the critical weaknesses raised by you:

SPhyR's entire framing is around its use as an environment for ...

If we would re-phrase the paper to be explicit about this being an evaluation environment, would that satisfy the raised weakness? Unfortunately, we might not have enough time or resources to train using this environment at this time.

SPhyR is a single task environment focused on topology optimisation ...

Agreed, we have one environment with various tasks and difficulty levels. Can you explain why it is a requirement to expand on the tasks for this to be a valuable contribution, we would appreciate this as this will change the way we will submit this work in the future.


Please can you conduct some RL training experiments with an appropriately sized model e.g. 4B-8B parameters?

Could a pass@k analysis act as a substitute for this (considering potential time and resource constraints)?


Once again, thank you very much for considering our work, we will consider your feedback carefully.

Reviewer_cBWwOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

SPhyR is an environment for spatial-physical reasoning. The model fills in the masked cells of a 2D grid with loads and supports, and the environment simulates the completion and scores it against a topology optimizer re-run on the masked region, so an equally efficient but different design scores as well as the reference. The benchmark comes in two variants. One is evaluated on eleven old LLMs, the other is regenerated at two grid sizes and evaluated on three recent LLMs.

Critical strengths

None

Critical weaknesses
  1. The submission does not read like a paper ready for review. It reads like experiment notes, a pile of bullet points and paragraph headings rather than a piece of writing with one line through it. From the paper, a first variant of the benchmark existed before this submission, and the paper itself says it could largely be solved by copying a neighbouring sample and that a high absolute score on it need not indicate reasoning about forces. A second variant is then generated to fix that. Yet the main table and the analyses that follow it still sit on the first variant, the second one gets a single table with three models, and the only thing learned from having both is that the second is harder. The models on the first variant are GPT-3.5, GPT-4o, Claude 3.7 Sonnet, Gemini 1.5 Pro and their 2025 contemporaries, which the paper calls contemporary. The appendix is dozens of pages of result dumps in a mess, with no table of contents. Information is scattered around, and none of it adds up to the main line a paper needs.

  2. The model panel is tested without a question behind it. Eleven models are run on the first variant and compared side by side, reasoning models next to non-reasoning ones, with no design that would let the ranking say anything about reasoning. The reasoning setting is accordingly mixed and unreported. DeepSeek-R1, Sonar Reasoning and Gemini 2.5 Pro reason, models like DeepSeek-V3 do not, whether extended thinking was on for the two Claude models is not stated, and temperature, token caps and reasoning switches should be given across all tested models. The one comparison with a purpose, R1 against V3 as the same base model without the reasoning training, shows what the panel should have been built from. A non-reasoning model belongs in the panel when it is the same model with reasoning off, and a ranking that mixes switched on and switched off models says little about reasoning.

Questions for authors
  1. The contribution "a reward that retrieval cannot reach" is argued from how the second variant was constructed, and no retrieval baseline number appears in the paper. What does a nearest-neighbour retrieval score on each variant?

  2. The title says environment and the abstract opens with reinforcement learning from verifiable rewards, yet no agent is trained and the conclusion leaves training to future work. What makes SPhyR an environment and not a benchmark at this point?

  3. In the feedback experiment the revision is told the material budget, as Section 4.1 says, while the control is shown nothing. How much of the gain remains when the control is also told the budget?

No offense here, but I would sincerely urge the authors to take responsibility for the work and submit a complete paper, since ICLR submissions are public online whether accepted or not. Suggestions follow.

  1. Make the second variant the benchmark. Rerun the main comparison and the analyses of Section 7 on it with current models and either drop the first variant or move it to an appendix. Otherwise even the insights found here will fade fast, since we are witnessing fast improvement of LLMs.

  2. Report the reasoning switch, effort and token cap for every model, and add an experiment that pairs every model with reasoning on and off.

1 comment

We want to thank the reviewer for their careful consideration of our work. Thank you for your efforts. We will carefully consider your feedback, but we are not confident we can implement the suggested changes within the given time frame for this submission. Thank you very much again and we believe your feedback will help us to make this manuscript a better submission next time.

Reviewer_mpXSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission proposes SPhyR, an environment for spatial-physical reasoning by LLMs. It builds on SIMP (Solid Isotropic Material with Penalization) topology optimization (Bendsøe & Sigmund, 2004), and it scores a design by simulating it rather than by comparing it with a stored answer.

The authors turn solved instances from an off-the-shelf SIMP plugin (Millipede for Grasshopper) into a first dataset (SPhyR-v1) of text grids, with letters marking load, support and masked cells. An LLM must fill every masked cell, with binary values in easy tasks and densities in hard tasks. Each completion is scored by a verifier that "solves it as a linear elastic structure and scores it against a topology optimizer re-run on exactly the masked region", and the resulting Topology Score (TS) combines stiffness relative to this reference with material use relative to the dataset answer's budget.

The environment follows the OpenEnv interface and can return feedback between attempts, without saying which cells were wrong. Because almost every v1 instance has a neighbour whose loads and supports differ by one cell, the authors regenerate the data (SPhyR-v2) with their own SIMP code, using rejection sampling that keeps instances apart.

No model is fine-tuned on the reward, and hosted LLMs such as GPT-4.1 (on v1) and Claude Opus 5 (on v2) are evaluated mainly zero-shot. Rankings by exact cell agreement and by TS can disagree, and trivial fills such as all-solid or uniform random score close to some LLMs. With feedback, an open-weight LLM (Qwen3.8) improves on most samples over a control that re-poses the same prompt without feedback.

Critical strengths
  • The verifier scores a partial design by simulating it. It compares the design with a reference re-optimized on exactly the masked cells at the same material budget. Scoring by simulated stiffness is established for full designs, as in TopoDiff (Mazé & Ahmed, 2023), and SPhyR extends it to completions. This shows that a design task without a unique answer can receive an automatic reward that does not require matching a stored answer, as Figure 1 illustrates.

  • The paper audits its own earlier release. It finds that v1 is a dense sweep in which almost every instance has a neighbour one cell away, usually with the same optimal structure (Section 8). It then regenerates the data as v2 by rejection sampling that keeps instances apart in loads, supports and solved structure, with ground truth from its own SIMP code. Others can therefore regenerate and extend the dataset from source, without the external solver.

  • The paper shows that scores on grid-output tasks partly measure the ability to emit a well-formed grid, and that this effect grows with grid size. At 20×20, ranking two models over all outputs and over well-formed grids only gives opposite orders (Section 8, Table 6 in the appendix). The paper therefore reports grid validity as its own metric and scores solid, void and random fills alongside its main results. These reporting practices could carry over to other benchmarks whose answers are serialised grids.

Critical weaknesses
  • SPhyR is presented as an environment for RL from verifiable rewards, but no model is fine-tuned on its reward, as CAD-Coder (Guan et al., 2025) does with GRPO on a geometric reward. The only evidence that the reward guides a model is a feedback experiment with one model on one task (Section 6). Its value as a training signal therefore remains open.

  • The Topology Score may leave little headroom. Solid and random fills score close to or above some models on both dataset versions (Tables 1 and 2). Completions that use the reference's material sometimes beat the reference, so the stiffness term saturates for them (Section 4.1). How well the score separates good designs from trivial ones is therefore unclear.

  • The abstract states that the reward "is not reachable by retrieval", but no baseline copies the nearest solved instance. The score drop from v1 to v2 also comes with changes to loads, supports, material budget and masking (Appendix E). The drop therefore cannot be attributed to the removal of neighbouring instances alone.

Overall, the submission appears to be a careful benchmark study, though perhaps not yet the RL environment it is presented as. It combines SIMP, finite-element analysis and the OpenEnv interface, and adds the masked-region reference, the v2 data and the analysis of measurement effects. The experiments show that the verifier ranks models differently from cell matching, but not that its reward can train a model.

Questions for authors
  1. Does the feedback gain show physical reasoning, or mainly that the model learns the material budget, which the single-step prompt does not state? Section 6 traces the gain to material use moving toward the reported budget, mostly in the first revision. Would the gain remain if both arms were told the budget upfront?

1.1. Do the prompts match the task? The revision prompt (Appendix D) always asks to "carry the load with less material", even when the feedback asks for more material. The task prompt (Figure 3) says "'V' indicates void.", while Section 3 uses V for masked cells.

  1. Which definitions of load carrying and of solid material do the metrics use? Section 4.1 sets the load-carrying threshold first against the fully solid grid, then against the reference design. Appendix B counts any nonzero density as solid for connectivity, while the tests in Appendix G require full density.

  2. Does Section 7 overstate how much more the interventions move cell agreement than structure? Under rotation, the cell-wise difference ratio rises or stays flat for three of five models (Table 10), while the Topology Score is said to fall by 11 to 20 points. No table reports these Topology Score values.

2 comments

Thank you very much for your time and for considering our work. We greatly appreciate your effort in going through our work. We agree with the weaknesses raised here and do not feel confident that we can make the adjustments needed in the given time. Once again, thank you very much, we will consider your feedback carefully and hope to submit a better manuscript next time.

Agreed that withdrawing, expanding, then re-submitting is likely the best option. The original ambition to develop challenging spatial-physical reasoning domains to evaluate RL from verifiable rewards is interesting. I hope that the authors can accomplish this by taking into account the reviewers' remarks. My review uses a reserved tone, while the other reviews use a more direct or critical tone. The criticism is likely aimed more at the experiments in this submission than at the idea and intent behind the work. Hence, I would like to encourage the authors to fully develop this benchmark.

Comments on the paper

1 comment

Dear authors,

The author-reviewer discussion period closes on November 18 (AoE). Please engage during this period so that we have our issues discussed thoroughly.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.